Related papers: Transfer Entropy and Directed Information in Gauss…
Transfer entropy is capable of capturing nonlinear source-destination relations between multi-variate time series. It is a measure of association between source data that are transformed into destination data via a set of linear…
Transfer Entropy, a generalisation of Granger Causality, promises to measure "information transfer" from a source to a target signal by ignoring self-predictability of a target signal when quantifying the source-target relationship. A…
Information estimates such as the ``direct method'' of Strong et al. (1998) sidestep the difficult problem of estimating the joint distribution of response and stimulus by instead estimating the difference between the marginal and…
Although an intimate relation between entropy and diffusion has been advocated for many years and even seems to have been verified in theory and experiments, a quantitatively reliable study, and any derivation of an algebraic relation…
Biochemical signaling cascades transmit intracellular information while dissipating energy under nonequilibrium conditions. We model a cascade as a code string and apply information-entropy ideas to quantify an optimal transmission rate. A…
We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual…
We consider a protocol for the two-time measurement of entropic observables in quantum open systems driven out of thermal equilibrium by coupling to several heat baths. We concentrate on the Markovian approximation of the time-evolution and…
Causal inference seeks to identify cause-and-effect interactions in coupled systems. A recently proposed method by Liang detects causal relations by quantifying the direction and magnitude of information flow between time series. The…
To infer information flow in any network of agents, it is important first and foremost to establish causal temporal relations between the nodes. Practical and automated methods that can infer causality are difficult to find, and the subject…
Motivated by entropic optimal transport, time reversal of diffusion processes is revisited. An integration by parts formula is derived for the carr\'e du champ of a Markov process in an abstract space. It leads to a time reversal formula…
We explore the dynamics of information systems. We show that the driving force for information dynamics is determined by both the information landscape and information flux which determines the equilibrium time reversible and the…
Using the formalism of information theory, we analyze the mechanism of information transduction in a simple one-step signaling cascade S$\rightarrow$X representing the gene regulatory network. Approximating the signaling channel to be…
A stochastic nonlinear dynamical system generates information, as measured by its entropy rate. Some---the ephemeral information---is dissipated and some---the bound information---is actively stored and so affects future behavior. We derive…
Fundamental relations between information and estimation have been established in the literature for the continuous-time Gaussian and Poisson channels, in a long line of work starting from the classical representation theorems by Duncan and…
We obtain the posterior distribution of a random process conditioned on observing the empirical frequencies of a finite sample path. We find under a rather broad assumption on the "dependence structure" of the process, {\em c.f.}…
Transfer entropy (TE) was introduced by Schreiber in 2000 as a measurement of the predictive capacity of one stochastic process with respect to another. Originally stated for discrete time processes, we expand the theory in line with recent…
Spiking activity from populations of neurons display causal interactions and memory effects. Therefore, they are expected to show some degree of irreversibility in time. Motivated by the spike train statistics, in this paper we build a…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
Shannon information entropy is a natural measure of probability (de)localization and thus (un)predictability in various procedures of data analysis for model systems. We pay particular attention to links between the Shannon entropy and the…
Markov categories are a novel framework to describe and treat problems in probability and information theory. In this work we combine the categorical formalism with the traditional quantitative notions of entropy, mutual information, and…