Related papers: Maximizing Multivariate Information with Error-Cor…
Since its introduction, the partial information decomposition (PID) has emerged as a powerful, information-theoretic technique useful for studying the structure of (potentially higher-order) interactions in complex systems. Despite its…
Multivariate circular observations, i.e. points on a torus are nowadays very common. Multivariate wrapped models are often appropriate to describe data points scattered on p-dimensional torus. However, statistical inference based on this…
Originally developed as a theory of consciousness, integrated information theory provides a mathematical framework to quantify the causal irreducibility of systems and subsets of units in the system. Specifically, mechanism integrated…
The total correlation(TC) is a crucial index to measure the correlation between marginal distribution in multidimensional random variables, and it is frequently applied as an inductive bias in representation learning. Previous research has…
Correlations play a pivotal role in various fields of science, particularly in quantum mechanics, yet their proper quantification remains a subject of debate. In this work, we aim to discuss the challenge of defining a reliable measure of…
A measure to estimate the direct and directional coupling in multivariate time series is proposed. The measure is an extension of a recently published measure of conditional Mutual Information from Mixed Embedding (MIME) for bivariate time…
This paper investigates the behavior of mutual information, entanglement negativity, and multipartite correlations in holographic RG flows, particularly during phase transitions. Mutual information provides a UV-finite measure of total…
In this paper, we examine a multi-sensor system where each sensor monitors multiple dynamic information processes and transmits updates over a shared communication channel. These updates may include correlated information across the various…
Mutual information (MI) is a promising candidate measure for the assessment and optimization of localization systems, as it captures nonlinear dependencies between random variables. However, the high cost of computing MI, especially for…
We propose new measures of shared information, unique information and synergistic information that can be used to decompose the multi-information of a pair of random variables $(Y,Z)$ with a third random variable $X$. Our measures are…
The correlation distance quantifies the statistical independence of two classical or quantum systems, via the distance from their joint state to the product of the marginal states. Tight lower bounds are given for the mutual information…
Informationally complete measurements allow the estimation of expectation values of any operator on a quantum system, by changing only the data-processing of the measurement outcomes. In particular, an informationally complete measurement…
Quantifying cooperation or synergy among random variables in predicting a single target random variable is an important problem in many complex systems. We review three prior information-theoretic measures of synergy and introduce a novel…
This paper provides a dual domain derivation of the error exponent of maximum mutual information (MMI) decoding with constant composition codes, showing it coincides with that of maximum likelihood decoding for discrete memoryless channels.…
Comparing clusterings is central to evaluating unsupervised models, yet the many existing similarity measures can produce widely divergent, sometimes contradictory, evaluations. Clustering similarity measures are typically organized into…
We consider an approximation scheme for multivariate information assuming that synergistic information only appearing in higher order joint distributions is suppressed, which may hold in large classes of systems. Our approximation scheme…
Deterministic chaos permits a precise notion of a "perfect measurement" as one that, when obtained repeatedly, captures all of the information created by the system's evolution with minimal redundancy. Finding an optimal measurement is…
Numerous studies have shown that multimodal LLMs process speech and images well but fail in non-intuitive ways rendering trivial tasks such as object counting unreliable. We investigate this behavior from an information-theoretic…
Quantum error correction protects quantum information against environmental noise. When using qubits, a measure of quality of a code is the maximum number of errors that it is able to correct. We show that a suitable notion of ``number of…
Maximal correlation is a measure of correlation for bipartite distributions. This measure has two intriguing features: (1) it is monotone under local stochastic maps; (2) it gives the same number when computed on i.i.d. copies of a pair of…