Related papers: Physics of the Shannon Limits
Shannon's notion of relative information between two physical systems can function as foundation for statistical mechanics and quantum mechanics, without referring to subjectivism or idealism. It can also represent a key missing element in…
In the theory of special relativity, energy can be found in two forms: kinetic energy and rest mass. Potential energy of a body is actually stored under the form of rest mass, interaction energy too, temperature is not. Information acquired…
We generalize the convex duality symmetry in Gibbs' statistical ensemble formulation, between Massieu's free entropy $\Phi_{V,N} (\beta)$ and the Gibbs entropy $\varphi_{V,N}(u)$ as a function of mean internal energy $u$. The duality tells…
Shannon based his information theory on the notion of probability measures as it we developed by Kolmogorov. In this paper we study some fundamental problems in information theory based on expectation measures. In the theory of expectation…
Whereas Shannon entropy is related to the growth rate of multinomial coefficients, we show that the quadratic entropy (Tsallis 2-entropy) is connected to their $q$-deformation; when $q$ is a prime power, these $q$-multinomial coefficients…
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…
We present quantitative relations between work and information that are valid both for finite sized and internally correlated systems as well in the thermodynamical limit. We suggest work extraction should be viewed as a game where the…
The link between information and energy introduces the observer and their knowledge into the understanding of a fundamental quantity of physics. Two approaches compete to account for this link, Brillouin's negentropy law of information and…
The detailed fluctuation theorem (DFT) is a statement about the asymmetry in the statistics of the entropy production. Consequences of the DFT are the second law of thermodynamics and the thermodynamics uncertainty relation (TUR), which…
Fluctuation theorems and the second law of thermodynamics are powerful relations constraining the behavior of out-of-equilibrium systems. While there exist generalizations of these relations to feedback controlled quantum systems, their…
Temperature is an outsider in the laws of motion given by Newton and Einstein and this oversight is the source of the predictions of time-reversal-invariance made by these two great systems of motion. By taking into consideration Planck's…
Remarkable progress of quantum information theory (QIT) allowed to formulate mathematical theorems for conditions that data-transmitting or data-processing occurs with a non-negative entropy gain. However, relation of these results…
Shannon's Entropy Power Inequality can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The entropy power inequality has played a key…
We expand upon a natural analogy between Bayesian statistics and statistical physics in which sample size corresponds to inverse temperature. This analogy motivates the definition of two novel statistical quantities: a learning capacity and…
We give a simple proof of the uncertainty principle with quantum side information, as in [Berta et al. Nature Physics 6, 659 (2010)], invoking the monotonicity of the relative entropy. Our proof shows that the entropic uncertainty principle…
We propose a new thermodynamic, relativistic relationship between information and entropy, which is closely analogous to the classic Maxwell electro-magnetic equations. Determination of whether information resides in points of…
In a previous work (M. Campisi. Stud. Hist. Phil. M. P. 36 (2005) 275-290) we have addressed the mechanical foundations of equilibrium thermodynamics on the basis of the Generalized Helmholtz Theorem. It was found that the volume entropy…
The uneven distribution of digits in numerical data, known as Benford's law, was discovered in 1881. Since then, this law has been shown to be correct in copious numerical data relating to economics, physics and even prime numbers. Although…
The second law of thermodynamics is the cornerstone of physics, characterizing the convertibility between thermodynamic states through a single function, entropy. Given the universal applicability of thermodynamics, a fundamental question…
The problem of the insensitivity of the macroscopic behavior of any thermodynamical system to partitioning generates a bias between the reproducibility of its macroscopic behavior viewed as the simplest form of causality and its long-term…