Related papers: A Dynamic Theory of Information and Entropy
We review of the interface between (theoretical) physics and information for non-experts. The origin of information as related to the notion of entropy is described, first in the context of thermodynamics then in the context of statistical…
Using a one-dimensional macromolecule in aqueous solution as an illustration, we demonstrate that the relative entropy from information theory, $\sum_k p_k\ln(p_k/p_k^*)$, has a natural role in the energetics of equilibrium and…
During the past three decades investigators have unveiled a number of deep connections between physical information and black holes whose consequences for ordinary systems go beyond what has been deduced purely from the axioms of…
First, the relation between black holes and limitations on information of other systems is developed. After reviewing the relation of entropy to information, we derive the entropy bound, review its applications to cosmology and its…
We present an information-theoretic assessment of atomic and molecular densities in the ground state and under a range of physical scenarios--excitation, confinement, and ensemblization. Comparisons across densities obtained from…
An essential role of information in microscopic thermodynamics (e.g. Maxwell's demon) opens a challenging question if there exists a formulation of the second law of thermodynamics based only on pure information ideas. Here, such a…
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a…
Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function \phi is employed as an analytic "device" to express information as…
The standard relations between information theory and thermodynamics are challenged. The Szilard engine is revisited and the bound proposed by Landauer is replaced by a different one which includes errors in information processing. Instead…
In this study, we present a reformulation of classical equilibrium thermodynamics by replacing the obscure and ambiguous concept of entropy with the clear and intuitive concept of information stored in a thermodynamic system. Specifically,…
Quantum mechanics and information theory are among the most important scientific discoveries of the last century. Although these two areas initially developed separately it has emerged that they are in fact intimately related. In this…
Information dynamics is an emerging description of information processing in complex systems which describes systems in terms of intrinsic computation, identifying computational primitives of information storage and transfer. In this paper…
We show that the conservation and the non-additivity of the information, together with the additivity of the entropy make the entropy increase in an isolated system. The collapse of the entangled quantum state offers an example of the…
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical…
Quantum theory reveals astonishing and counterintuitive phenomena not found in classical physics, such as wave-particle duality, where entities like electrons and photons exhibit both wave-like and particle-like behaviors. In this paper, we…
Inspired by foundational studies in classical and quantum physics, and by information retrieval studies in quantum information theory, we prove that the notions of 'energy' and 'entropy' can be consistently introduced in human language and,…
We have presented first an axiomatic derivation of Boltzmann entropy on the basis of two axioms consistent with two basic properties of thermodynamic entropy. We have then studied the relationship between Boltzmann entropy and information…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…