Related papers: On the Dynamic Statistical Information Theory
Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function \phi is employed as an analytic "device" to express information as…
We propose a new thermodynamic, relativistic relationship between information and entropy, which is closely analogous to the classic Maxwell electro-magnetic equations. Determination of whether information resides in points of…
The dissipation phenomena of relative entropy from an It\^o--Langevin dynamical system is a classic topic from stochastic analysis. Relying on the time-reversal of diffusions, a novel trajectorial approach investigates the pathwise behavior…
The gap in statistics between multi-variate and time-series analysis can be bridged by using entropy statistics and recent developments in multi-dimensional scaling. For explaining the evolution of the sciences as non-linear dynamics, the…
Biological organisms adapt to changes by processing informations from different sources, most notably from their ancestors and from their environment. We review an approach to quantify these informations by analyzing mathematical models of…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…
Biochemical signaling cascades transmit intracellular information while dissipating energy under nonequilibrium conditions. We model a cascade as a code string and apply information-entropy ideas to quantify an optimal transmission rate. A…
We theoretically investigate how information flows when two particles interact with each other. Understanding the physical mechanisms of directional information flow is crucial for advancing information thermodynamics and stochastic…
This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error…
In many complex systems, whether biological or artificial, the thermodynamic costs of communication among their components are large. These systems also tend to split information transmitted between any two components across multiple…
In this paper we discuss about the validity of the Shannon entropy functional in connection with the correct Gibbs-Hertz probability distribution function. We show that there is no contradiction in using the Shannon-Gibbs functional and…
The concept of entropy in nonequilibrium macroscopic systems is investigated in the light of an extended equation of motion for the density matrix obtained in a previous study. It is found that a time-dependent information entropy can be…
Many biological phenomena or social events critically depend on how information evolves in complex networks. However, a general theory to characterize information evolution is yet absent. Consequently, numerous unknowns remain about the…
Transfer Entropy and Directed Information are information-theoretic measures of the directional dependency between stochastic processes. Following the definitions of Schreiber and Massey in discrete time, we define and evaluate these…
Information theoretic measures (entropies, entropy rates, mutual information) are nowadays commonly used in statistical signal processing for real-world data analysis. The present work proposes the use of Auto Mutual Information (Mutual…
Information theory is a statistical theory concerned with the relative state of detectors and physical systems. As a consequence, the classical framework of Shannon needs to be extended to deal with quantum detectors, possibly moving at…
This special issue collects contributions from the participants of the "Information in Dynamical Systems and Complex Systems" workshop, which cover a wide range of important problems and new approaches that lie in the intersection of…
Understanding the mutual interdependence between the behavior of dynamical processes on networks and the underlying topologies promises new insight for a large class of empirical networks. We present a generic approach to investigate this…
Biological, artificial, and physical systems dissipate energy to accurately transmit information. While tools of information theory have been used to characterize information-processing capabilities, how reliably this information is…