Related papers: Occam's Razor Cuts Away the Maximum Entropy Princi…
A vast concourse of events and phenomena occur in nature that may be interrelated by a entropy-maximization technique that provides a comprehensible explanation of a range of physical problems, integrating in a new framework the universal…
The uncertainty principle can be expressed in entropic terms, also taking into account the role of entanglement in reducing uncertainty. The information exclusion principle bounds instead the correlations that can exist between the outcomes…
After the justification of the maximum entropy principle for equilibrium mechanical system from the principle of virtual work, i.e., the virtual work of microscopic forces on the elements of a mechanical system vanishes in thermodynamic…
The claim that life is an emergent phenomenon exhibiting novel properties and principles is often criticized for being in conflict with causal closure at the microscopic level. I argue that advances in cosmological theory suggesting an…
The maximum entropy principle (MEP) is a method for obtaining the most likely distribution functions of observables from statistical systems, by maximizing entropy under constraints. The MEP has found hundreds of applications in ergodic and…
The Shannon-Khinchin axioms for the ordinary information entropy are generalized in a natural way to the nonextensive systems based on the concept of nonextensive conditional entropy, and a complete proof of the uniqueness theorem for the…
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth…
Different quantities that go by the name of entropy are used in variational principles to infer probability distributions from limited data. Shore and Johnson showed that maximizing the Boltzmann- Gibbs form of the entropy ensures that…
A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…
We generalize to non equilibrium states Onsager's minimum dissipation principle. We also interpret this principle and some previous results in terms of optimal control theory. Entropy production plays the role of the cost necessary to drive…
We deal with a generalized statistical description of nonequilibrium complex systems based on least biased distributions given some prior information. A maximum entropy principle is introduced that allows for the determination of the…
In this article we provide initial findings regarding the problem of solving likelihood equations by means of a maximum entropy approach. Unlike standard procedures that require equating at zero the score function of the maximum-likelihood…
It is shown that Onsager's principle of the least dissipation of energy is equivalent to the maximum entropy production principle. It is known that solutions of the linearized Boltzmann equation make extrema of entropy production. It is…
We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…
We give a simple proof of the uncertainty principle with quantum side information, as in [Berta et al. Nature Physics 6, 659 (2010)], invoking the monotonicity of the relative entropy. Our proof shows that the entropic uncertainty principle…
The universe is certainly not yet in total thermodynamical equilibrium,so clearly some law telling about special initial conditions is needed. A universe or a system imposed to behave periodically gets thereby required ``initial…
Based on the form invariance of the structures given by Khinchin's axiomatic foundations of information theory and the pseudoadditivity of the Tsallis entropy indexed by q, the concept of conditional entropy is generalized to the case of…
The maximum entropy ansatz, as it is often invoked in the context of time-series analysis, suggests the selection of a power spectrum which is consistent with autocorrelation data and corresponds to a random process least predictable from…
Experimental and theoretical results about entropy limits for macroscopic and single-particle systems are reviewed. It is clarified when it is possible to speak about a quantum of entropy, given by the Boltzmann constant k, and about a…
Entropy is a very useful concept from physics that tries to explain how a system behaves from a point of view of the thermodynamics. However, there are two ways to explain entropy, and it depends on if we are studying a microsystem or a…