Related papers: Characterising information-theoretic storage and t…
Understanding the temporal dependence of precipitation is key to improving weather predictability and developing efficient stochastic rainfall models. We introduce an information-theoretic approach to quantify memory effects in discrete…
An information theoretic measure is derived that quantifies the statistical coherence between systems evolving in time. The standard time delayed mutual information fails to distinguish information that is actually exchanged from shared…
Whether the system under study is a shoal of fish, a collection of neurons, or a set of interacting atmospheric and oceanic processes, transfer entropy measures the flow of information between time series and can detect possible causal…
Current methods for pattern analysis in time series mainly rely on statistical features or probabilistic learning and inference methods to identify patterns and trends in the data. Such methods do not generalize well when applied to…
The deep connection between entropy and information is discussed in terms of both classical and quantum physics. The mechanism of information transfer between systems via entanglement is explored in the context of decoherence theory. The…
Understanding information processing in the brain requires the ability to determine the functional connectivity between the different regions of the brain. We present a method using transfer entropy to extract this flow of information…
A cornerstone of our understanding of both biological and artificial neural networks is that they store information in the strengths of synaptic connections among the neurons. However, in contrast to the well-established theory for…
A central task in analyzing complex dynamics is to determine the loci of information storage and the communication topology of information flows within a system. Over the last decade and a half, diagnostics for the latter have come to be…
Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation…
We consider the formalism of information decomposition of target effects from multi-source interactions, i.e. the problem of defining redundant and synergistic components of the information that a set of source variables provides about a…
In this work, we investigate the relation between the concept of ``information rate'', an information geometric method for measuring the speed of the time evolution of the statistical states of a stochastic process, and stochastic…
Natural memories are associative, declarative and distributed. Symbolic computing memories resemble natural memories in their declarative character, and information can be stored and recovered explicitly; however, they lack the associative…
The field of complex networks studies a wide variety of interacting systems by representing them as networks. To understand their properties and mutual relations, the randomisation of network connections is a commonly used tool. However,…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
The information diffusion prediction on social networks aims to predict future recipients of a message, with practical applications in marketing and social media. While different prediction models all claim to perform well, general…
Brain activity differs vastly between sleep, cognitive tasks, and action. Information theory is an appropriate concept to analytically quantify these brain states. Based on neurophysiological recordings, this concept can handle complex data…
We consider discrete stochastic processes, modeled by classical master equations, on networks. The temporal growth of the lack of information about the system is captured by its non-equilibrium entropy, defined via the transition…
Information theory establishes the fundamental limits on data transmission, storage, and processing. Quantum information theory unites information theoretic ideas with an accurate quantum-mechanical description of reality to give a more…
Learning is based on synaptic plasticity, which affects and is driven by neural activity. Because pre- and postsynaptic spiking activity is shaped by randomness, the synaptic weights follow a stochastic process, requiring a probabilistic…
Renewal processes are broadly used to model stochastic behavior consisting of isolated events separated by periods of quiescence, whose durations are specified by a given probability law. Here, we identify the minimal sufficient statistic…