Related papers: Total/dual correlation/coherence, redundancy/syner…
The emergent dynamics of complex systems often arise from the internal dynamical interactions among different elements and hence is to be modeled using multiple variables that represent the different dynamical processes. When such systems…
We present a discussion of generalized statistics based on Renyi's, Fisher's and Tsallis's measures of information. The unifying conceptual framework which we employ here is provided by information theory. Important applications of…
We address the problem of efficiently and informatively quantifying how multiplets of variables carry information about the future of the dynamical system they belong to. In particular we want to identify groups of variables carrying…
In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that…
Current orthogonal matching pursuit (OMP) algorithms calculate the correlation between two vectors using the inner product operation and minimize the mean square error, which are both suboptimal when there are non-Gaussian noises or…
While information theory has been introduced to characterize the fundamental limitations of control and filtering for a few decades, the existing information-theoretic methods are indirect and cumbersome for analyzing the limitations of…
We introduce an information theoretic measure of statistical structure, called 'binding information', for sets of random variables, and compare it with several previously proposed measures including excess entropy, Bialek et al.'s…
Context dependence is central to the description of complexity. Keying on the pairwise definition of "set complexity" we use an information theory approach to formulate general measures of systems complexity. We examine the properties of…
The apparent dichotomy between information-processing and dynamical approaches to complexity science forces researchers to choose between two diverging sets of tools and explanations, creating conflict and often hindering scientific…
Exploiting the theory of state space models, we derive the exact expressions of the information transfer, as well as redundant and synergistic transfer, for coupled Gaussian processes observed at multiple temporal scales. All of the terms,…
We introduce an information-theoretic method for quantifying causality in chaotic systems. The approach, referred to as IT-causality, quantifies causality by measuring the information gained about future events conditioned on the knowledge…
We propose a novel measure to assess the presence of meso-scale structures in complex networks. This measure is based on the identification of regular patterns in the adjacency matrix of the network, and on the calculation of the quantity…
Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation…
We introduce a class of information measures based on group entropies, allowing us to describe the information-theoretical properties of complex systems. These entropic measures are nonadditive, and are mathematically deduced from a series…
Complex high-dimensional co-occurrence data are increasingly popular from a complex system of interacting physical, biological and social processes in discretely indexed modifiable areal units or continuously indexed locations of a study…
Entropy governs molecular self-assembly, phase transitions, and material stability, yet remains challenging to quantify and directly control in molecular systems. Here, we demonstrate that the computable information density (CID), a data…
Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…
The asymptotic mutual information (MI) analysis for multiple-input multiple-output (MIMO) systems over double-scattering channels has achieved engaging results, but the convergence rates of the mean, variance, and the distribution of the MI…
Pairwise metrics are often employed to estimate statistical dependencies between brain regions, however they do not capture higher-order information interactions. It is critical to explore higher-order interactions that go beyond paired…
Assessing the synergistic high-order behaviors (HOBs) that emerge from underlying structural mechanisms is crucial to characterize complex systems. This work leverages the combined use of predictability and information measures to detect…