Related papers: Relative Entropy in Biological Systems
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
Many thermodynamic relations involve inequalities, with equality if a process does not involve dissipation. In this article we provide equalities in which the dissipative contribution is shown to involve the relative entropy (a.k.a.…
Many thermodynamic relations involve inequalities, with equality if a process does not involve dissipation. In this article we provide equalities in which the dissipative contribution is shown to involve the relative entropy (a.k.a.…
The population dynamics and stability of ecosystems of interacting species is studied from the perspective of non-equilibrium thermodynamics by assuming that species, through their biotic and abiotic interactions, are units of entropy…
We analyze a contrasting dynamical behavior of Gibbs-Shannon and conditional Kullback-Leibler entropies, induced by time-evolution of continuous probability distributions. The question of predominantly purpose-dependent entropy definition…
Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical…
Entropy is one of the key thermodynamic variables reflecting changes in the state of matter. Unlike other thermodynamic variables, it is well-defined also for nonequilibrium steady states through its relation to information. Applying this…
The concept of entropy in nonequilibrium macroscopic systems is investigated in the light of an extended equation of motion for the density matrix obtained in a previous study. It is found that a time-dependent information entropy can be…
The maximum entropy formalism developed by Jaynes determines the relevant ensemble in nonequilibrium statistical mechanics by maximising the entropy functional subject to the constraints imposed by the available information. We present an…
The second law of thermodynamics states that entropy production in macroscopic systems is non-negative, reaching zero only at thermodynamic equilibrium. As a corollary, this implies that the state trajectory of macroscopic systems is…
The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields…
To describe the nonequilibrium states of a system we introduce a new thermodynamic parameter - the lifetime of a system. The statistical distributions which can be obtained out of the mesoscopic description characterizing the behaviour of a…
The second law of thermodynamics states that the entropy of an isolated system can only increase over time. This appears to conflict with the reversible evolution of isolated quantum systems under the Schr\"odinger equation, which preserves…
The evolution of entropy is derived with respect to dynamical systems. For a stochastic system, its relative entropy $D$ evolves in accordance with the second law of thermodynamics; its absolute entropy $H$ may also be so, provided that the…
The second law of thermodynamics governs that nonequilibrium systems evolve towards states of higher entropy over time. However, it does not specify the rate of this evolution and the role of fluctuations that impact the system's dynamics.…
Using a one-dimensional macromolecule in aqueous solution as an illustration, we demonstrate that the relative entropy from information theory, $\sum_k p_k\ln(p_k/p_k^*)$, has a natural role in the energetics of equilibrium and…
We give a simple proof of the uncertainty principle with quantum side information, as in [Berta et al. Nature Physics 6, 659 (2010)], invoking the monotonicity of the relative entropy. Our proof shows that the entropic uncertainty principle…
Despite the importance of the Second Law of Thermodynamics, it is not absolute. Statistical mechanics implies that, given sufficient time, systems near equilibrium will spontaneously fluctuate into lower-entropy states, locally reversing…
We analyze dynamical properties of the Shannon information entropy of a continuous probability distribution, which is driven by a standard diffusion process. This entropy choice is confronted with another option, employing the conditional…
Entropy serves as a central observable in equilibrium thermodynamics. However, many biological and ecological systems operate far from thermal equilibrium. Here we show that entropy production can characterize the behavior of such…