Related papers: Transfer Entropy: where Shannon meets Turing
Transfer entropy has been used to quantify the directed flow of information between source and target variables in many complex systems. While transfer entropy was originally formulated in discrete time, in this paper we provide a framework…
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
Transfer entropy (TE) is an attractive model-free method to detect causality and infer structural connectivity of general digital systems. However it relies on high dimensions used in its definition to clearly remove the memory effect and…
We propose a method for computing the transfer entropy between time series using Ulam's approximation of the Perron-Frobenius (transfer) operator associated with the map generating the dynamics. Our method differs from standard transfer…
We propose a formal expansion of the transfer entropy to put in evidence irreducible sets of variables which provide information for the future state of each assigned target. Multiplets characterized by a large contribution to the expansion…
Entropy is a fundamental concept in quantum information theory that allows to quantify entanglement and investigate its properties, for example its monogamy over multipartite systems. Here, we derive variational formulas for relative…
We explore the connection between deep learning and information theory through the paradigm of diffusion models. A diffusion model converts noise into structured data by reinstating, imperfectly, information that is erased when data was…
We derive a functional for the entropy contributed by any microscopic degrees of freedom as arising from their measurable pair correlations. Applicable both in and out of equilibrium, this functional yields the maximum entropy which a…
The ability to quantify the directional flow of information is vital to understanding natural systems and designing engineered information-processing systems. A widely used measure to quantify this information flow is the transfer entropy.…
Similarity-sensitive entropy measures the uncertainty of a probability law relative to a similarity kernel that encodes the distinguishability between states. We develop a measure-theoretic treatment covering both finite similarity matrices…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
Uncertainty relations provide constraints on how well the outcomes of incompatible measurements can be predicted, and, as well as being fundamental to our understanding of quantum theory, they have practical applications such as for…
We introduce a novel entropy-related function, \textit{non-repeatability}, designed to capture dynamical behaviors in complex systems. Its normalized form, \textit{mutability}, has been previously applied in statistical physics as a…
A recently proposed history formalism is used to define temporal entanglement in quantum systems, and compute its entropy. The procedure is based on the time-reduction of the history density operator, and allows a symmetrical treatment of…
Given a set of heterogeneous source datasets with their classifiers, how can we quickly find the most useful source dataset for a specific target task? We address the problem of measuring transferability between source and target datasets,…
Entanglement is a key quantum phenomena and understanding transitions between phases of matter with different entanglement properties are an interesting probe of quantum mechanics. We numerically study a model of a 2D tensor network…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
With the help of transfer entropy, we analyze information flows between communities of complex networks. We show that the transfer entropy provides a coherent description of interactions between communities, including non-linear…
Inferring the directionality of interactions between cellular processes is a major challenge in systems biology. Time-lagged correlations allow to discriminate between alternative models, but they still rely on assumed underlying…