Related papers: On the Dynamic Statistical Information Theory
An interactive stochastics, evaluated by an entropy functional (EF) of a random field and informational process' path functional (IPF), allows us modeling the evolutionary information processes and revealing regularities of evolution…
The concept of entropy rate for a dynamical process on a graph is introduced. We study diffusion processes where the node degrees are used as a local information by the random walkers. We describe analitically and numerically how the degree…
Analysis of a probabilistic system often requires to learn the joint probability distribution of its random variables. The computation of the exact distribution is usually an exhaustive precise analysis on all executions of the system. To…
Recent experiments have indicated that many biological systems self-organise near their critical point, which hints at a common design principle. While it has been suggested that information transmission is optimized near the critical…
Thermodynamics is based on the notions of energy and entropy. While energy is the elementary quantity governing physical dynamics, entropy is the fundamental concept in information theory. In this work, starting from first principles, we…
We calculate Shannon information entropy of trapped interacting bosons in both the position and momentum spaces, $S_r$ and $S_k$ respectively. The total entropy maintains the fuctional form $S=a + b \ln N$ for repulsive bosons. At the…
The dynamics of many-body systems can often be captured in terms of only a few relevant variables. Mathematical and numerical approaches exist to identify these variables by exploiting a separation of time scales between slow relevant and…
Statistical divergences are important tools in data analysis, information theory, and statistical physics, and there exist well known inequalities on their bounds. However, in many circumstances involving temporal evolution, one needs…
Recent research has explored the increasingly important role of social media by examining the dynamics of individual and group behavior, characterizing patterns of information diffusion, and identifying influential individuals. In this…
The question of deriving general force/flux relationships that apply out of the linear response regime is a central topic of theories for nonequilibrium statistical mechanics. This work applies an information theory perspective to compute…
Shannon based his information theory on the notion of probability measures as it we developed by Kolmogorov. In this paper we study some fundamental problems in information theory based on expectation measures. In the theory of expectation…
So far, feedback-driven systems have been discussed using (i) measurement and control, (ii) a tape interacting with a system or (iii) by identifying an implicit Maxwell demon in steady state transport. We derive the corresponding second…
Traditional statistical estimation, or statistical inference in general, is static, in the sense that the estimate of the quantity of interest does not change the future evolution of the quantity. In some sequential estimation problems…
In this work, a strategy to estimate the information transfer between the elements of a complex system, from the time series associated to the evolution of this elements, is presented. By using the nearest neighbors of each state, the local…
I introduce a new approach to semantic information based upon the influence of erasure operations (interventions) upon distributions of a system's future trajectories through its phase space. Semantic (meaningful) information is…
Irreversible processes accomplished in a fixed time involve nonlinearly coupled flows of matter, energy, and information. Here, using entropy production as an example, we show how thermodynamic uncertainty relations and speed limits on…
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…
A central result that arose in applying information theory to the stochastic thermodynamics of nonlinear dynamical systems is the Information-Processing Second Law (IPSL): the physical entropy of the universe can decrease if compensated by…
Denoising diffusion models have spurred significant gains in density modeling and image generation, precipitating an industrial revolution in text-guided AI art generation. We introduce a new mathematical foundation for diffusion models…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…