Related papers: Maximizing Multivariate Information with Error-Cor…
Testing the independence between random vectors is a fundamental problem in statistics. Distance correlation, a recently popular dependence measure, is universally consistent for testing independence against all distributions with finite…
When a large body of data from diverse experiments is analyzed using a theoretical model with many parameters, the standard error matrix method and the general tools for evaluating errors may become inadequate. We present an iterative…
We derive upper bounds on the generalization error of a learning algorithm in terms of the mutual information between its input and output. The bounds provide an information-theoretic understanding of generalization in learning problems,…
Joint measurements of non-commuting observables are characterized by unavoidable measurement uncertainties that can be described in terms of the error statistics for input states with well-defined values for the target observables. However,…
Based on the notion of maximal correlation, Kimeldorf, May and Sampson (1980) introduce a measure of correlation between two random variables, called the "concordant monotone correlation" (CMC). We revisit, generalize and prove new…
This study confirms a local trade-off between information and disturbance in quantum measurements. It is represented by the correlation between the changes in these two quantities when the measurement is slightly modified. The correlation…
Selective rationalization improves neural network interpretability by identifying a small subset of input features -- the rationale -- that best explains or supports the prediction. A typical rationalization criterion, i.e. maximum mutual…
Many recent methods for unsupervised or self-supervised representation learning train feature extractors by maximizing an estimate of the mutual information (MI) between different views of the data. This comes with several immediate…
The connection between index coding and matroid theory have been well studied in the recent past. El Rouayheb et al. established a connection between multilinear representation of matroids and wireless index coding. Muralidharan and Rajan…
Estimating mutual information (MI) is a fundamental task in data science and machine learning. Existing estimators mainly rely on either highly flexible models (e.g., neural networks), which require large amounts of data, or overly…
Using the relative entropy of total correlation, we derive an expression relating the mutual information of $n$-partite pure states to the sum of the mutual informations and entropies of its marginals and analyze some of its implications.…
Estimating mutual information between continuous random variables is often intractable and extremely challenging for high-dimensional data. Recent progress has leveraged neural networks to optimize variational lower bounds on mutual…
Motivated by thermodynamic considerations, we analyse the variation of the quantum mutual information on a unitary orbit of a bipartite system's state, with and without global constraints such as energy conservation. We solve the full…
Generalization error bounds are essential to understanding machine learning algorithms. This paper presents novel expected generalization error upper bounds based on the average joint distribution between the output hypothesis and each…
In many scientific tasks we are interested in discovering whether there exist any correlations in our data. This raises many questions, such as how to reliably and interpretably measure correlation between a multivariate set of attributes,…
We introduce a new information theoretic measure of quantum correlations for multiparticle systems. We use a form of multivariate mutual information -- the interaction information and generalize it to multiparticle quantum systems. There…
Measures of dependence among variables, and measures of information content and shared information have become valuable tools of multi-variable data analysis. Information measures, like marginal entropies, mutual and multi-information, have…
Estimating mutual correlations between random variables or data streams is essential for intelligent behavior and decision-making. As a fundamental quantity for measuring statistical relationships, mutual information has been extensively…
Traditional measures based solely on pairwise associations often fail to capture the complex statistical structure of multivariate data. Existing approaches for identifying information shared among $d>3$ variables are frequently…
The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions $\Pi_{\mathcal{A}}$, on small enough…