Related papers: The many faces of multivariate information
Information-theoretic quantities, such as entropy, are used to quantify the amount of information a given variable provides. Entropies can be used together to compute the mutual information, which quantifies the amount of information two…
The paper makes the observation that all orders of information entropy are equal in signals composed of repeating units of distinct symbols where the units can be classified as a member of a symmetry group. This leads to an improved metric…
The problems of causality, modeling, and control for chaotic, high-dimensional dynamical systems are formulated in the language of information theory. The central quantity of interest is the Shannon entropy, which measures the amount of…
A core feature of complex systems is that the interactions between elements in the present causally constrain each-other as the system evolves through time. To fully model all of these interactions (between elements, as well as ensembles of…
Higher order interactions are increasingly recognised as a fundamental aspect of complex systems ranging from the brain to social contact networks. Hypergraph as well as simplicial complexes capture the higher-order interactions of complex…
The decomposition of channel information into synergies of different order is an open, active problem in the theory of complex systems. Most approaches to the problem are based on information theory, and propose decompositions of mutual…
A Holevo measure is used to discuss how much information about a given POVM on system $a$ is present in another system $b$, and how this influences the presence or absence of information about a different POVM on $a$ in a third system $c$.…
This paper describes a method for identification of the informative variables in the information system with discrete decision variables. It is targeted specifically towards discovery of the variables that are non-informative when…
Real collective density variables $C(\boldsymbol{k})$ [c.f. Eq.\ref{Equation3})] in many-particle systems arise from non-linear transformations of particle positions, and determine the structure factor $S(\boldsymbol{k})$, where $\bf k$…
We introduce a method to learn a hierarchy of successively more abstract representations of complex data based on optimizing an information-theoretic objective. Intuitively, the optimization searches for a set of latent factors that best…
Collective organization in matter plays a significant role in its expressed physical properties. Typically, it is detected via an order parameter, appropriately defined for each given system's observed emergent patterns. Recent developments…
While the standard network description of complex systems is based on quantifying links between pairs of system units, higher-order interactions (HOIs) involving three or more units play a major role in governing the collective network…
Information Theory provides a fundamental basis for analysis, and for a variety of subsequent methodological approaches, in relation to uncertainty quantification. The transversal character of concepts and derived results justifies its…
Interaction information is one of the multivariate generalizations of mutual information, which expresses the amount information shared among a set of variables, beyond the information, which is shared in any proper subset of those…
Mutually unbiased bases, mutually unbiased measurements and general symmetric informationally complete measurements are three related concepts in quantum information theory. We investigate multipartite systems using these notions and…
The aim is formal principles of origin information and information process creating information observer self-creating information in interactive observations. The interactive phenomenon creates Yes-No actions of information Bits in its…
We propose an information-theoretic quantifier for the advantage gained from cooperation that captures the degree of dependency between subsystems of a global system. The quantifier is distinct from measures of multipartite correlations…
Higher-order networks effectively represent complex systems with group interactions. Existing methods usually overlook the relative contribution of group interactions (hyperlinks) of different sizes to the overall network structure. Yet,…
Networks are a fundamental tool for understanding and modeling complex systems in physics, biology, neuroscience, engineering, and social science. Many networks are known to exhibit rich, lower-order connectivity patterns that can be…
Models that rely solely on pairwise relationships often fail to capture the complete statistical structure of the complex multivariate data found in diverse domains, such as socio-economic, ecological, or biomedical systems. Non-trivial…