Related papers: On Entropy Minimization and Convergence
The Gibbs sampler (a.k.a. Glauber dynamics and heat-bath algorithm) is a popular Markov Chain Monte Carlo algorithm which iteratively samples from the conditional distributions of a probability measure $\pi$ of interest. Under the…
Given n independent Bernoulli(p) random variables X_i, i = 1, ..., n, representing the opinions of individuals connected by an underlying random k-regular graph G_n on {1, ..., n}, we show that when conditioned on an atypical empirical…
We develop the argument that the Gibbs-von Neumann entropy is the appropriate statistical mechanical generalisation of the thermodynamic entropy, for macroscopic and microscopic systems, whether in thermal equilibrium or not, as a…
A broad set of sufficient conditions that guarantees the existence of the maximum entropy (maxent) distribution consistent with specified bounds on certain generalized moments is derived. Most results in the literature are either focused on…
In its continuous version, the entropy functional measuring the information content of a given probability density may be plagued by a "measure" problem that results from improper weighting of phase space. This issue is addressed…
The thermodynamic limit of the internal energy and the entropy of the system of quantum interacting particles in random medium is shown to exist under the crucial requirements of stability and temperedness of interactions. The energy turns…
The organization of high-dimensional probability spaces is a fundamental problem at the intersection of statistical physics and information theory. Here, we analyze the distributions populating level surfaces of the probability simplex…
The entropy shows an unavoidable tendency of disorder in thermostatistics according to the second thermodynamics law. This provides a minimization entropy principle for quantum thermostatistics with the von Neumann entropy and nonextensive…
We seek an entropy estimator for discrete distributions with fully empirical accuracy bounds. As stated, this goal is infeasible without some prior assumptions on the distribution. We discover that a certain information moment assumption…
Microcanonical thermodynamics allows the application of statistical mechanics both to finite and even small systems and also to the largest, self-gravitating ones. However, one must reconsider the fundamental principles of statistical…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
For more than 100 years, one of the central concepts in statistical mechanics has been the microcanonical ensemble, which provides a way of calculating the thermodynamic entropy for a specified energy. A controversy has recently emerged…
The probability distribution of a function of a subsystem conditioned on the value of the function of the whole, in the limit when the ratio of their values goes to zero, has a limit law: It equals the unconditioned marginal probability…
When at equilibrium, large-scale systems obey conventional thermodynamics because they belong to microscopic configurations (or states) that are typical. Crucially, the typical states usually represent only a small fraction of the total…
We demonstrate that the Gibbs-Shannon entropy is applicable to non-equilibrium systems of any size and boundary conditions. The change in microscopic entropy can be attributed to the stochastic nature of dynamic processes and to the…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
In the previous papers (Kui\'{c} et al. in Found Phys 42:319-339, 2012; Kui\'{c} in arXiv:1506.02622, 2015), it was demonstrated that applying the principle of maximum information entropy by maximizing the conditional information entropy,…
The phenomenon of entropy concentration provides strong support for the maximum entropy method, MaxEnt, for inferring a probability vector from information in the form of constraints. Here we extend this phenomenon, in a discrete setting,…
For a quantum system in a macroscopically large volume $V$, prepared in a pure state and subject to maximally noisy or ergodic unitary dynamics, the reduced density matrix of any sub-system $v\ll V$ is almost surely totally mixed. We show…
We give some results relating asymptotic characterisations of maximum entropy probability measures to characterisations of Bayes optimal classifiers. Our main theorems show that maximum entropy is a universally Bayes optimal decision rule…