Related papers: On Thermodynamic Information
Maximum entropy principle identifies forces conjugated to observables and the thermodynamic relations between them, independent upon their underlying mechanistic details. For data about state distributions or transition statistics, the…
In view of the relation between information and thermodynamics we investigate how much information about an external protocol can be stored in the memory of a stochastic measurement device given an energy budget. We consider a layered…
Shannon Information theory has achieved great success in not only communication technology where it was originally developed for but also many other science and engineering fields such as machine learning and artificial intelligence.…
The problems of causality, modeling, and control for chaotic, high-dimensional dynamical systems are formulated in the language of information theory. The central quantity of interest is the Shannon entropy, which measures the amount of…
Given a universe of discourse X-a domain of possible outcomes-an experiment may consist of selecting one of its elements, subject to the operation of chance, or of observing the elements, subject to imprecision. A priori uncertainty about…
At this point in time, two major areas of physics, statistical mechanics and quantum mechanics, rest on the foundations of probability and entropy. The last century saw several significant fundamental advances in our understanding of the…
We discuss a basic thermodynamic properties of systems with multifractal structure. This is possible by extending the notion of Gibbs-Shannon's entropy into more general framework - Renyi's information entropy. We show a connection of…
In classical phenomenological thermodynamics the first and second laws can be regarded as independent statements. Statistical mechanics provides a microscopic substratum that explains thermodynamics in probabilistic terms via a microstate…
We have formulated a family of machine learning problems as the time evolution of Parametric Probabilistic Models (PPMs), inherently rendering a thermodynamic process. Our primary motivation is to leverage the rich toolbox of thermodynamics…
By considering an explicit nonequilibrium model, we analyze the statistics of the irreversible work, $w_{\rm irr}$, and irreversible entropy production, $\Delta_i s$, within the stochastic energetics framework. Restating the second law of…
The Free-Energy-Principle (FEP) is an influential and controversial theory which postulates a deep and powerful connection between the stochastic thermodynamics of self-organization and learning through variational inference. Specifically,…
Fisher information and Shannon entropy are fundamental tools for understanding and analyzing dynamical systems from complementary perspectives. They can characterize unknown parameters by quantifying the information contained in variables,…
We present a validation of the asdf method, an information-theoretic framework for computing thermodynamic entropy from molecular configurations. The method reformulates entropy estimation as the Shannon entropy of a residual mapping…
We review and investigate the general theory of thermodynamics of computation, and derive the fundamental inequalities that set the lower bounds of the work requirement and the heat emission during a computation. These inequalities…
Shannon entropy was defined for probability distributions and then its using was expanded to measure the uncertainty of knowledge for systems with complete information. In this article, it is proposed to extend the using of Shannon entropy…
Second law of thermodynamics can be apparently violated for systems whose dynamics depends on acquired information by measurement. However, when one consider measurement and erasure process together along with the system it saves the second…
Recently, a thermodynamic definition of time has been introduced. This definition is useful to find approach some open problems in physics. But, it was obtained by a phenomenological approach and a logical inconsistency appears in the…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
We recall some of the history of the information-theoretic approach to deriving core results in probability theory and indicate parts of the recent resurgence of interest in this area with current progress along several interesting…
Statistical divergences are important tools in data analysis, information theory, and statistical physics, and there exist well known inequalities on their bounds. However, in many circumstances involving temporal evolution, one needs…