Related papers: Can Transfer Entropy Infer Information Flow in Neu…

Information theory is introduced in this lecture note with a particular emphasis on its relevance to algebraic coding theory. The document develops the mathematical foundations for quantifying uncertainty and information transmission by…

Measures of the direction and strength of the interdependence between two time series are evaluated and modified in order to reduce the bias in the estimation of the measures, so that they give zero values when there is no causal effect.…

The problems of causality, modeling, and control for chaotic, high-dimensional dynamical systems are formulated in the language of information theory. The central quantity of interest is the Shannon entropy, which measures the amount of…

Dynamic effective connectivity networks (dECNs) reveal the changing directed brain activity and the dynamic causal influences among brain regions, which facilitate the identification of individual differences and enhance the understanding…

Information flow between components of a system takes many forms and is key to understanding the organization and functioning of large-scale, complex systems. We demonstrate three modalities of information flow from time series X to time…

Ever since Claude Shannon used entropy for his "Mathematical Theory of Communication", entropy has become a buzzword in research circles with scientists applying entropy to describe any phenomena that are reminiscent of disorder. In this…

We theoretically investigate the flow of information in an interacting two-skyrmion system confined in a box at finite temperature. By numerical simulations based on the Thiele-Langevin equation, we demonstrate that the skyrmion motion…

In classical information theory, a causal relationship between two variables is typically modelled by assuming that, for every possible state of one of the variables, there exists a particular distribution of states of the second variable.…

Brain connectivity characterizes interactions between different regions of a brain network during resting-state or performance of a cognitive task. In studying brain signals such as electroencephalograms (EEG), one formal approach to…

We show that a rate of conditional Shannon entropy reduction, characterizing the learning of an internal process about an external process, is bounded by the thermodynamic entropy production. This approach allows for the definition of an…

Coordinated responses to environmental stimuli are critical for multicellular organisms. To overcome the obstacles of cell-to-cell heterogeneity and noisy signaling dynamics within individual cells, cells must effectively exchange…

Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…

Transfer entropy is used to establish a measure of causal relationships between two variables. Symbolic transfer entropy, as an estimation method for transfer entropy, is widely applied due to its robustness against non-stationarity. This…

We develop new algorithms for estimating heterogeneous treatment effects, combining recent developments in transfer learning for neural networks with insights from the causal inference literature. By taking advantage of transfer learning,…

Transferring knowledge from one neural network to another has been shown to be helpful for learning tasks with few training examples. Prevailing fine-tuning methods could potentially contaminate pre-trained features by comparably high…

We investigate causal inference in the asymptotic regime as the number of variables approaches infinity using an information-theoretic framework. We define structural entropy of a causal model in terms of its description complexity measured…

Time lags are ubiquitous in biophysiological processes and more generally in real-world complex networks. It has been recently proposed to use information-theoretic tools such as transfer entropy to detect and estimate a possible delay in…

Without having direct access to the information that is being exchanged, traces of information flow can be obtained by looking at temporal sequences of user interactions. These sequences can be represented as causality trees whose…

Convolutional Neural Networks (CNNs) have become the foundation of modern computer vision, achieving unprecedented accuracy across diverse image recognition tasks. While these networks excel on in-distribution data, they remain vulnerable…

We propose a constraint-based algorithm, which automatically determines causal relevance thresholds, to infer causal networks from data. We call these topological thresholds. We present two methods for determining the threshold: the first…