Related papers: Transfer Entropy Rate Through Lempel-Ziv Complexit…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…
We introduce a new measure to evaluate the transferability of representations learned by classifiers. Our measure, the Log Expected Empirical Prediction (LEEP), is simple and easy to compute: when given a classifier trained on a source data…
We propose a formal expansion of the transfer entropy to put in evidence irreducible sets of variables which provide information for the future state of each assigned target. Multiplets characterized by a large contribution to the expansion…
The Cross Entropy method is a well-known adaptive importance sampling method for rare-event probability estimation, which requires estimating an optimal importance sampling density within a parametric class. In this article we estimate an…
Causal discovery is a fundamental problem in statistics and has wide applications in different fields. Transfer Entropy (TE) is a important notion defined for measuring causality, which is essentially conditional Mutual Information (MI).…
Physics concepts have often been borrowed and independently developed by other fields of science. In this perspective a significant example is that of entropy in Information Theory. The aim of this paper is to provide a short and…
Transfer entropy (TE) is an attractive model-free method to detect causality and infer structural connectivity of general digital systems. However it relies on high dimensions used in its definition to clearly remove the memory effect and…
In this work, a strategy to estimate the information transfer between the elements of a complex system, from the time series associated to the evolution of this elements, is presented. By using the nearest neighbors of each state, the local…
The concept of entropy rate for a dynamical process on a graph is introduced. We study diffusion processes where the node degrees are used as a local information by the random walkers. We describe analitically and numerically how the degree…
How does the information flow between different brain regions during various stimuli? This is the question we aim to address by studying complex cognitive paradigms in terms of Information Theory. To assess creativity and the emergence of…
This article gives a self-contained analysis of the performance of the Lempel-Ziv compression algorithm on (hidden) Markovian sources. Specifically we include a full proof of the assertion that the compression rate approaches the entropy…
A central task in analyzing complex dynamics is to determine the loci of information storage and the communication topology of information flows within a system. Over the last decade and a half, diagnostics for the latter have come to be…
Information entropy is applied to the analysis of time series generated by dynamical systems. Complexity of a temporal or spatio-temporal signal is defined as the difference between the sum of entropies of the local linear regions of the…
We present a measure of local information transfer, derived from an existing averaged information-theoretical measure, namely transfer entropy. Local transfer entropy is used to produce profiles of the information transfer into each…
Here, we propose a new tool to estimate the complexity of a time series: the entropy of difference (ED). The method is based solely on the sign of the difference between neighboring values in a time series. This makes it possible to…
Characterizing the entropy of a system is a crucial, and often computationally costly, step in understanding its thermodynamics. It plays a key role in the study of phase transitions, pattern formation, protein folding and more. Current…
Complexity measures are essential to understand complex systems and there are numerous definitions to analyze one-dimensional data. However, extensions of these approaches to two or higher-dimensional data, such as images, are much less…
We analyze emission trajectories from a driven-dissipative two-qubit system and a classical telegraph model with matched rates. Using Lempel-Ziv complexity, mutual information, and temporal correlations, we show that both models undergo a…
We propose a method for computing the transfer entropy between time series using Ulam's approximation of the Perron-Frobenius (transfer) operator associated with the map generating the dynamics. Our method differs from standard transfer…