Related papers: Generalized Common Informations: Measuring Commonn…
The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model…
Measuring the relationship between any pair of variables is a rich and active area of research that is central to scientific practice. In contrast, characterizing the common information among any group of variables is typically a…
In this paper we consider the axiomatic characterization of information and certainty measures in a unified way. We present the general axiomatic system which captures the common properties of a large number of the measures previously…
Consider two correlated sources $X$ and $Y$ generated from a joint distribution $p_{X,Y}$. Their G\'acs-K\"orner Common Information, a measure of common information that exploits the combinatorial structure of the distribution $p_{X,Y}$,…
The problem of preserving privacy when a multivariate source is required to be revealed partially to multiple users is modeled as a Gray-Wyner source coding problem with K correlated sources at the encoder and K decoders in which the kth…
Correlations disguised in various forms underlie a host of important phenomena in classical and quantum systems, such as information and energy exchanges. The quantum mutual information and the norm of the correlation matrix are both…
We study the generation of a secret key of maximum rate by a pair of terminals observing correlated sources and with the means to communicate over a noiseless public com- munication channel. Our main result establishes a structural…
Based on the notion of maximal correlation, Kimeldorf, May and Sampson (1980) introduce a measure of correlation between two random variables, called the "concordant monotone correlation" (CMC). We revisit, generalize and prove new…
We presented assisted common information as a generalization of G\'acs-K\"orner (GK) common information at ISIT 2010. The motivation for our formulation was to improve upperbounds on the efficiency of protocols for secure two-party sampling…
A new bimodal generative model is proposed for generating conditional and joint samples, accompanied with a training method with learning a succinct bottleneck representation. The proposed model, dubbed as the variational Wyner model, is…
We present a new notion $R_\ell$ of higher-order common information, which quantifies the information that $\ell\geq 2$ arbitrarily distributed random variables have in common. We provide analytical lower bounds on $R_3$ and $R_4$ for…
We show the mutual information between the targets in a Gray-Wyner Network as a bound that separates Wyner's lossy common information and G\'acs-K\"orner lossy common information. The results are a generalization of the lossless case…
A privacy-constrained information extraction problem is considered where for a pair of correlated discrete random variables $(X,Y)$ governed by a given joint distribution, an agent observes $Y$ and wants to convey to a potentially public…
The question of how much communication is required between collaborating parties to compute a function of their data is of fundamental importance in the fields of theoretical computer science and information theory. In this work, the focus…
The Pearson correlation, correlation ratio, and maximal correlation have been well-studied in the literature. In this paper, we study the conditional versions of these quantities. We extend the most important properties of the unconditional…
We provide an information-theoretic framework for studying the generalization properties of machine learning algorithms. Our framework ties together existing approaches, including uniform convergence bounds and recent methods for adaptive…
We propose two novel extensions of the Wyner common information optimization problem. Each relaxes one fundamental constraints in Wyner's formulation. The \textit{Variational Wyner Common Information} relaxes the matching constraint to the…
We define On-Average KL-Privacy and present its properties and connections to differential privacy, generalization and information-theoretic quantities including max-information and mutual information. The new definition significantly…
In this correspondence, we correct an erroneous result on the achievability part of the R\'enyi common information with order $1+s\in(1,2]$ in [1]. The new achievability result (upper bound) of the R\'enyi common information no longer…
Secure multi-party computation is a central problem in modern cryptography. An important sub-class of this are problems of the following form: Alice and Bob desire to produce sample(s) of a pair of jointly distributed random variables. Each…