Related papers: Maximizing Multivariate Information with Error-Cor…
Multivariate datasets are common in various real-world applications. Recently, copulas have received significant attention for modeling dependencies among random variables. A copula-based information measure is required to quantify the…
Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual…
We study formal properties of smooth max-information, a generalization of von Neumann mutual information derived from the max-relative entropy. Recent work suggests that it is a useful quantity in one-shot channel coding, quantum rate…
Information-theoretic quantities reveal dependencies among variables in the structure of joint, marginal, and conditional entropies, but leave some fundamentally different systems indistinguishable. Furthermore, there is no consensus on how…
We live in unprecedented times in terms of our ability to use evidence to inform medical care. For example, we can perform data-driven post-test probability calculations. However, there is work to do. As has been previously noted,…
We study multivariate problems like function approximation, numerical integration, global optimization and dispersion. We obtain new results on the information complexity $n(\varepsilon,d)$ of these problems. The information complexity is…
This paper considers the problem of recovering a structured signal from a relatively small number of noisy measurements with the aid of a similar signal which is known beforehand. We propose a new approach to integrate prior information…
One of the most fundamental questions one can ask about a pair of random variables X and Y is the value of their mutual information. Unfortunately, this task is often stymied by the extremely large dimension of the variables. We might hope…
We provide matching upper and lower bounds on the mutual information in noisy reconstruction of parity check codes and thereby prove a long-standing conjecture by Montanari [IEEE Transactions on Information Theory 2005]. Besides extending a…
We study the problem of distributed mean estimation and optimization under communication constraints. We propose a correlated quantization protocol whose leading term in the error guarantee depends on the mean deviation of data points…
The mutual and tripartite information between pairs and triples of disjoint regions in a quantum field theory are sensitive probes of the spread of correlations in an equilibrating system. We compute these quantities in strongly coupled…
We aim to discover manipulation concepts embedded in the unannotated demonstrations, which are recognized as key physical states. The discovered concepts can facilitate training manipulation policies and promote generalization. Current…
In this letter we determine the derivative of the mutual information corresponding to bit-interleaved coded modulation systems. The derivative follows as a linear combination of minimum-mean-squared error functions of coded modulation sets.…
The problem of model selection is considered for the setting of interpolating estimators, where the number of model parameters exceeds the size of the dataset. Classical information criteria typically consider the large-data limit,…
Is perfect error correction always worth the trouble? A framework is presented for the analysis of error detection and correction in multi-level systems of communication that takes into account degrees of freedom attended and ignored by…
The ability of information processing in biologically motivated Boolean networks is of interest in recent information theoretic research. One measure to quantify this ability is the well known mutual information. Using Fourier analysis we…
Multi-view learning is frequently used in data science. The pairwise correlation maximization is a classical approach for exploring the consensus of multiple views. Since the pairwise correlation is inherent for two views, the extensions to…
This article focuses on measurement error in covariates in regression analyses in which the aim is to estimate the association between one or more covariates and an outcome, adjusting for confounding. Error in covariate measurements, if…
Information distance is a parameter-free similarity measure based on compression, used in pattern recognition, data mining, phylogeny, clustering, and classification. The notion of information distance is extended from pairs to multiples…
A method is discussed that allows combining sets of differential or inclusive measurements. It is assumed that at least one measurement was obtained with simultaneously fitting a set of nuisance parameters, representing sources of…