Related papers: Combinatorial Entropies and Statistics
It is well known that a Shannon based definition of information entropy leads in the classical case to the Boltzmann entropy. It is tempting to regard the Von Neumann entropy as the corresponding quantum mechanical definition. But the…
Counting ad infinitum is the holographic observable to a statistical dynamics with finite states under independent repeated sampling. Entropy provides the infinitesimal probability for an observed frequency $\hat{\boldsymbol{\nu}}$ w.r.t. a…
Revision of the paper previously entitled "Learning a Machine for the Decision in a Partially Observable Markov Universe" In this paper, we are interested in optimal decisions in a partially observable universe. Our approach is to directly…
Given a universe of discourse X-a domain of possible outcomes-an experiment may consist of selecting one of its elements, subject to the operation of chance, or of observing the elements, subject to imprecision. A priori uncertainty about…
Non-extensive systems do not allow to go to the thermodynamic limit. Therefore we have to reformulate statistical mechanics without invoking the thermodynamical limit. I.e. we have to go back to Pre-Gibbsian times. We show that Boltzmann's…
We make precise sense of the idea of "molecular chaos" through algorithmic randomness of microscopic trajectories, and ground macroscopic irreversibility in the lack of symmetry under time reversal of this property. This concept of…
Entropy notions for $\varepsilon$-incremental practical stability and incremental stability of deterministic nonlinear systems under disturbances are introduced. The entropy notions are constructed via a set of points in state space which…
In this paper, I expand Shannon's definition of entropy into a new form of entropy that allows integration of information from different random events. Shannon's notion of entropy is a special case of my more general definition of entropy.…
As physics searches for invariants in observations, this paper looks for invariants of probabilistic observation without assuming physical structure. Structure emerges from the basic assumption of science that new information shall lead to…
Works, briefly surveyed here, are concerned with two basic methods: Maximum Probability and Bayesian Maximum Probability; as well as with their asymptotic instances: Relative Entropy Maximization and Maximum Non-parametric Likelihood.…
Learning the parameters of graphical models using the maximum likelihood estimation is generally hard which requires an approximation. Maximum composite likelihood estimations are statistical approximations of the maximum likelihood…
A composite likelihood is an inference function derived by multiplying a set of likelihood components. This approach provides a flexible framework for drawing inference when the likelihood function of a statistical model is computationally…
Statistics of distinguishable particles has become relevant in systems of colloidal particles and in the context of applications of statistical mechanics to complex networks. When studying these type of systems with the standard textbook…
The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of…
Estimation of Distribution Algorithms (EDAs) require flexible probability models that can be efficiently learned and sampled. Deep Boltzmann Machines (DBMs) are generative neural networks with these desired properties. We integrate a DBM…
In this paper, we compute the tightest possible bounds on the probability that the optimal value of a combinatorial optimization problem in maximization form with a random objective exceeds a given number, assuming only knowledge of the…
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from…
The thermal-equilibrium canonical distribution is currently obtained by maximizing the Boltzmann-Gibbs-von Neumann-Shannon entropy $S_{BG}(p)=k\sum^{W}_{i=1}p_{i}\ln 1/p_{i}$ constrained to $\sum^{W}_{i=1}p_{i}=1$ and…
The ultimate purpose of the statistical analysis of ordinal patterns is to characterize the distribution of the features they induce. In particular, knowing the joint distribution of the pair Entropy-Statistical Complexity for a large class…
Entropic uncertainty relations, based on sums of entropies of probability distributions arising from different measurements on a given pure state, can be seen as a generalization of the Heisenberg uncertainty relation that is in many cases…