Related papers: Physics of the Shannon Limits
Energy has no definition, except that given by a conservation principle which essentially amounts to defining it as the elements of an open list of unknown cardinality. Entropy, identified by Shannon as information we lack, has too many…
A thermodynamic framework for asymptotic inference is developed in which sample size and parameter variance define a state space. Within this description, Shannon information plays the role of entropy, and an integrating factor organizes…
In this paper, we analyze the relationship between entropy and information in the context of the mixing process of two identical ideal gases. We will argue that entropy has a special information-based feature that is enfolded in the…
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits…
The essence of the second law of classical thermodynamics is the `entropy principle' which asserts the existence of an additive and extensive entropy function, S, that is defined for all equilibrium states of thermodynamic systems and whose…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
This article is a short version of a longer article to appear in Physics Reports (cond-mat/9708200). The essential postulates of classical thermodynamics are formulated, from which the second law is deduced as the principle of increase of…
Traditional form of the second law of thermodynamics is strongly restricted by three conditions: One is the initial joint state of the system and surroundings should be a product state, so that there exists no initial correlations. The…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
We extend present Shannon's static statistical information theory to dynamic processes and establish a dynamic statistical information theory. We derive the nonlinear evolution equations of dynamic information density and dynamic…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
A general nonequilibrium thermodynamic theory is developed for time-dependent Langevin dynamics, starting from the common definition of nonequilibrium Gibbs entropy. It is shown that the notations appearing in the First and the Second Law…
The Gibbs distribution universally characterizes states of thermal equilibrium. In order to extend the Gibbs distribution to non-equilibrium steady states, one must relate the self-information $\mathcal{I}(x) = -\log(P_\text{ss}(x))$ of…
This article introduces the physics of information in the context of molecular biology and genomics. Entropy and information, the two central concepts of Shannon's theory of information and communication, are often confused with each other…
We review of the interface between (theoretical) physics and information for non-experts. The origin of information as related to the notion of entropy is described, first in the context of thermodynamics then in the context of statistical…
This paper is a non-technical, informal presentation of our theory of the second law of thermodynamics as a law that is independent of statistical mechanics and that is derivable solely from certain simple assumptions about adiabatic…
Statistical mechanics descriptions of the second law of thermodynamics generally imply point-like particles driven by a dissipative overall mechanism for their simultaneous time-evolution. As the number of involved particles grows larger,…
We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and…
The second law of thermodynamics, which deals with irreversibility and makes the theory so special, is usually considered empirical. The definition of equilibrium as an attractor, on the other hand, requires a postulate. This article shows…
A microscopic definition of the thermodynamic entropy in an isolated quantum system must satisfy (i) additivity, (ii) extensivity and (iii) the second law of thermodynamics. We show that the diagonal entropy, which is the Shannon entropy in…