Related papers: Polyadic Entropy, Synergy and Redundancy among Sta…
Identifying the origin of nonequilibrium characteristics in a generic interacting system having multiple degrees of freedom is a challenging task. In this context, information theoretic measures such as mutual information and related…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
Understanding a complex system entails capturing the non-trivial collective phenomena that arise from interactions between its different parts. Information theory is a flexible and robust framework to study such behaviours, with several…
This brief note considers the problem of estimating temporal synergy and integrated information in dyadic dynamical processes. One of the standard estimators of dynamic synergy is based on the minimal mutual information between sets of…
Information theoretic measures (entropies, entropy rates, mutual information) are nowadays commonly used in statistical signal processing for real-world data analysis. The present work proposes the use of Auto Mutual Information (Mutual…
The aim of this paper is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures that based on mutual information, for a class of structured spiking neuronal network. In…
Entropy is one of the key thermodynamic variables reflecting changes in the state of matter. Unlike other thermodynamic variables, it is well-defined also for nonequilibrium steady states through its relation to information. Applying this…
We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual…
Stochastic forces in natural systems are rarely isotropic. From hydrodynamically coupled colloids to chemical reaction networks, noise contributions are inherently correlated. Together with internal interactions and changing environments,…
The interactions between three or more random variables are often nontrivial, poorly understood, and yet, are paramount for future advances in fields such as network information theory, neuroscience, genetics and many others. In this work,…
In this paper, we analyze the relationship between entropy and information in the context of the mixing process of two identical ideal gases. We will argue that entropy has a special information-based feature that is enfolded in the…
Obtaining meaningful quantitative descriptions of the statistical dependence within multivariate systems is a difficult open problem. Recently, the Partial Information Decomposition (PID) was proposed to decompose mutual information (MI)…
Information dynamics is an emerging description of information processing in complex systems which describes systems in terms of intrinsic computation, identifying computational primitives of information storage and transfer. In this paper…
The concept of entropy in nonequilibrium macroscopic systems is investigated in the light of an extended equation of motion for the density matrix obtained in a previous study. It is found that a time-dependent information entropy can be…
This Thesis explores how tools from Statistical Physics and Information Theory can help us describe and understand complex systems. In the first part, we study the interplay between internal interactions, environmental changes, and…
In statistical physics entropy is usually introduced as a global quantity which expresses the amount of information that would be needed to specify the microscopic configuration of a system. However, for lattice models with infinitely many…
We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter…
We propose a unified theoretical framework for quantifying spatio-temporal interactions in a stochastic dynamical system based on information geometry. In the proposed framework, the degree of interactions is quantified by the divergence…
An information theoretic measure is derived that quantifies the statistical coherence between systems evolving in time. The standard time delayed mutual information fails to distinguish information that is actually exchanged from shared…