Related papers: Transfer Entropy and Directed Information in Gauss…

We study the multiple definitions of the entropy production for discrete-time Markov processes in single systems and composite systems. These definitions have been studied in single systems, but less so in composite systems. With a clear…

Stimulated by the need of describing useful notions related to information measures, we introduce the `pdf-related distributions'. These are defined in terms of transformation of absolutely continuous random variables through their own…

There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…

In this paper we develop the concept of information transfer between the Borel-measurable sets for a dynamical system described by a measurable space and a non-singular transformation. The concept is based on how Shannon entropy is…

We investigate the role of Massey's directed information in portfolio theory, data compression, and statistics with causality constraints. In particular, we show that directed information is an upper bound on the increment in growth rates…

Hidden information emerges under impulse interactions with Markov diffusion process modeling interactive random environment. Impulse yes no action cuts Markov correlations revealing Bit of hidden information connected correlated states.…

It is well known that semantic and structural features of the generated images emerge at different times during the reverse dynamics of diffusion, a phenomenon that has been connected to physical phase transitions in magnets and other…

For a Markovian dynamics on discrete states, the logarithmic ratio of waiting-time distributions between two successive, instantaneous transitions in forward and backward direction is a measure of time-irreversibility. It thus serves as an…

Information theory is an outstanding framework to measure uncertainty, dependence and relevance in data and systems. It has several desirable properties for real world applications: it naturally deals with multivariate data, it can handle…

We formulate a new model for transport in stochastic media with long-range spatial correlations where exponential attenuation (controlling the propagation part of the transport) becomes power law. Direct transmission over optical distance…

Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…

Systems driven away from thermal equilibrium constantly deliver entropy to their environment. Determining this entropy production requires detailed information about the system's internal states and dynamics. However, in most practical…

Relative entropy is a powerful measure of the dissimilarity between two statistical field theories in the continuum. In this work, we study the relative entropy between Gaussian scalar field theories in a finite volume with different masses…

Information dynamics is an emerging description of information processing in complex systems which describes systems in terms of intrinsic computation, identifying computational primitives of information storage and transfer. In this paper…

Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide…

The act of measuring a system has profound consequences of dynamical and thermodynamic nature. In particular, the degree of irreversibility ensuing from a non-equilibrium process is strongly affected by measurements aimed at acquiring…

We propose a partial information decomposition based on the newly introduced framework of causal tensors, i.e., multilinear stochastic maps that transform source data into destination data. This framework enables us to express an indirect…

Entropy, its production, and its change in a dynamical system can be understood from either a fully stochastic dynamic description or from a deterministic dynamics exhibiting chaotic behavior. By taking the former approach based on the…

We introduce an epistemic information measure between two data streams, that we term $influence$. Closely related to transfer entropy, the measure must be estimated by epistemic agents with finite memory resources via sampling accessible…

Time plays an essential role in the diffusion of information, influence and disease over networks. In many cases we only observe when a node copies information, makes a decision or becomes infected -- but the connectivity, transmission…