Related papers: Expanding the Transfer Entropy to Identify Informa…
Transfer entropy (TE) captures the directed relationships between two variables. Partial transfer entropy (PTE) accounts for the presence of all confounding variables of a multivariate system and infers only about direct causality. However,…
We develop information-theoretic measures of spatial structure and pattern in more than one dimension. As is well known, the entropy density of a two-dimensional configuration can be efficiently and accurately estimated via a converging…
Transfer Entropy and Directed Information are information-theoretic measures of the directional dependency between stochastic processes. Following the definitions of Schreiber and Massey in discrete time, we define and evaluate these…
The transfer entropy is a well-established measure of information flow, which quantifies directed influence between two stochastic time series and has been shown to be useful in a variety fields of science. Here we introduce the transfer…
We define the class of multivariate group entropies as a novel set of information - theoretical measures, which extends significantly the family of group entropies. We propose new examples related to the "super-exponential" universality…
We propose a new estimator to measure directed dependencies in time series. The dimensionality of data is first reduced using a new non-uniform embedding technique, where the variables are ranked according to a weighted sum of the amount of…
Estimating information-theoretic quantities such as entropy and mutual information is central to many problems in statistics and machine learning, but challenging in high dimensions. This paper presents estimators of entropy via inference…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
Information theory allows us to investigate information processing in neural systems in terms of information transfer, storage and modification. Especially the measure of information transfer, transfer entropy, has seen a dramatic surge of…
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical…
Transfer entropy is an established method for quantifying directed statistical dependencies in neuroimaging and complex systems datasets. The pairwise (or bivariate) transfer entropy from a source to a target node in a network does not…
Transfer entropy, an information-theoretic measure of time-directed information transfer between joint processes, has steadily gained popularity in the analysis of complex stochastic dynamics in diverse fields, including the neurosciences,…
According to E.T. Jaynes and E.P. Wigner, entropy is an anthropomorphic concept in the sense that in a physical system correspond many thermodynamic systems. The physical system can be examined from many points of view each time examining…
The description of the dynamics of complex systems, in particular the capture of the interaction structure and causal relationships between elements of the system, is one of the central questions of interdisciplinary research. While the…
Dissipative quantum systems are frequently described within the framework of the so-called "system-plus-reservoir" approach. In this work we assign their description to the Maximum Entropy Formalism and compare the resulting thermodynamic…
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…
Complex systems produce high-dimensional signals that lack macroscopic variables analogous to entropy, temperature, or free energy. This work introduces a thermoinformational formulation that derives entropy, internal energy, temperature,…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
Inference of causality is central in nonlinear time series analysis and science in general. A popular approach to infer causality between two processes is to measure the information flow between them in terms of transfer entropy. Using…
We propose the Fourier-domain transfer entropy spectrum, a novel generalization of transfer entropy, as a model-free metric of causality. For arbitrary systems, this approach systematically quantifies the causality among their different…