Related papers: Entropy Concentration and the Empirical Coding Gam…
What is information? Is it physical? We argue that in a Bayesian theory the notion of information must be defined in terms of its effects on the beliefs of rational agents. Information is whatever constrains rational beliefs and therefore…
We introduce a new generalization of relative entropy to non-negative vectors with sums $\gt 1$. We show in a purely combinatorial setting, with no probabilistic considerations, that in the presence of linear constraints defining a convex…
Finding the minimal relative entropy of two quantum states under semidefinite constraints is a pivotal problem located at the mathematical core of various applications in quantum information theory. An efficient method for providing…
In the dynamics of open quantum systems, the backflow of information to the reduced system under study has been suggested as the actual physical mechanism inducing memory and thus leading to non-Markovian quantum dynamics. To this aim, the…
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example.…
We show that under natural growth conditions on the entropy function, convergence in relative entropy is equivalent to $L_p$-convergence. The main tool is the theory of Young measures, in a form that accounts for the formation of…
We introduce a framework to study discrete-variable (DV) quantum systems based on qudits. It relies on notions of a mean state (MS), a minimal stabilizer-projection state (MSPS), and a new convolution. Some interesting consequences are: The…
We consider a generic system composed of a fixed number of particles distributed over a finite number of energy levels. We make only general assumptions about system's properties and the entropy. System's constraints other than fixed number…
We show how to determine the maximum and minimum possible values of one measure of entropy for a given value of another measure of entropy. These maximum and minimum values are obtained for two standard forms of probability distribution (or…
Statistical mechanics is based on interplay between energy minimization and entropy maximization. Here we formalize this interplay via axioms of cooperative game theory (Nash bargaining) and apply it out of equilibrium. These axioms capture…
We propose a method to derive the stationary size distributions of a system, and the degree distributions of networks, using maximisation of the Gibbs-Shannon entropy. We apply this to a preferential attachment-type algorithm for systems of…
The Renyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or…
We make use of an entropic property to establish a convergence theorem (Main Theorem), which reveals that the conditional entropy measures the asymptotic Gaussianity. As an application, we establish the {\it entropic conditional central…
This work is concerned with the minimization of quantum entropies under local constraints of density, current, and energy. The problem arises in the work of Degond and Ringhofer about the derivation of quantum hydrodynamical models from…
When at equilibrium, large-scale systems obey conventional thermodynamics because they belong to microscopic configurations (or states) that are typical. Crucially, the typical states usually represent only a small fraction of the total…
In this paper we describe a triple correspondence between graph limits, information theory and group theory. We put forward a new graph limit concept called log-convergence that is closely connected to dense graph limits but its main…
Entropies of mixing can be derived directly from the parent distributions of extreme value theory. They correspond to pseudo-additive entropies in the case of Pareto and power function distributions, while to the Shannon entropy in the case…
What is the relationship between plausibility logic and the principle of maximum entropy? When does the principle give unreasonable or wrong results? When is it appropriate to use the rule `expectation = average'? Can plausibility logic…
This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal…
The problems of conditional entropy's definition and the formula to compute conditional entropy are analyzed from various perspectives, and the corrected computing formula is presented. Examples are given to prove the conclusion that…