Related papers: On Thermodynamic Information
The information-thermodynamics link is revisited, going back to the analysis of Szilard's engine. It is argued that instead of equivalence rather complementarity of physical entropy and information theoretical one is a correct concept.…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
An information theoretic approach inspired by quantum statistical mechanics was recently proposed as a means to optimize network models and to assess their likelihood against synthetic and real-world networks. Importantly, this method does…
From the principle of maximum entropy for a closed system in thermal equilibrium, for the first instance a clear relation is shown to exist between total entropy S (in terms of arrangements of particles) and the classical expression for the…
We expand upon a natural analogy between Bayesian statistics and statistical physics in which sample size corresponds to inverse temperature. This analogy motivates the definition of two novel statistical quantities: a learning capacity and…
A system responding to a stochastic driving signal can be interpreted as computing, by means of its dynamics, an implicit model of the environmental variables. The system's state retains information about past environmental fluctuations,…
The presence of symmetries imposes a stringent set of constraints on a system. This constrained structure allows intelligent agents interacting with such a system to drastically improve the efficiency of learning and generalization, through…
We present a pedagogical review of the fundamental concepts in thermodynamics of information, by focusing on the second law of thermodynamics and the entropy production. Especially, we discuss the relationship among thermodynamic…
Solar vortex regions show enhanced Poynting flux and favourable heating conditions, but how the vortices reorganize and influence their surroundings remains unclear. Here we apply information-theoretic diagnostics to a Bifrost simulation to…
A novel formalism, called H-theory, is applied to the problem of statistical equilibrium of a hierarchical complex system with multiple time and length scales. In this approach, the system is formally treated as being composed of a small…
In our former work [K. Tadaki, Local Proceedings of CiE 2008, pp.425-434, 2008], we developed a statistical mechanical interpretation of algorithmic information theory by introducing the notion of thermodynamic quantities at temperature T,…
Statistical mechanics is based on interplay between energy minimization and entropy maximization. Here we formalize this interplay via axioms of cooperative game theory (Nash bargaining) and apply it out of equilibrium. These axioms capture…
Entanglement measures based on a logarithmic functional form naturally emerge in any attempt to quantify the degree of entanglement in the state of a multipartite quantum system. These measures can be regarded as generalizations of the…
Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…
Understanding a complex system entails capturing the non-trivial collective phenomena that arise from interactions between its different parts. Information theory is a flexible and robust framework to study such behaviours, with several…
Statistical thermodynamics is valuable as a conceptual structure that shapes our thinking about equilibrium thermodynamic states. A cloud of unresolved questions surrounding the foundations of the theory could lead an impartial observer to…
We generalize the convex duality symmetry in Gibbs' statistical ensemble formulation, between Massieu's free entropy $\Phi_{V,N} (\beta)$ and the Gibbs entropy $\varphi_{V,N}(u)$ as a function of mean internal energy $u$. The duality tells…
The selection of an equilibrium state by maximising the entropy of a system, subject to certain constraints, is often powerfully motivated as an exercise in logical inference, a procedure where conclusions are reached on the basis of…
We examine the relationship between the Bayesian and information-theoretic formulations of source separation algorithms. This work makes use of the relationship between the work of Claude E. Shannon and the "Recent Contributions" by Warren…
The $(i)$ reciprocity relations for the relative Fisher information (RFI, hereafter) and $(ii)$ a generalized RFI-Euler theorem, are self-consistently derived from the Hellmann-Feynman theorem. These new reciprocity relations generalize the…