Related papers: Lower bound on Wyner's Common Information

The most natural way to describe an information-carrying system containing a specific noise is an additive white Gaussian-noise (AWGN) channel. In bosonic quantum systems (especially the Gaussian case), although the classical information…

We derive general upper bounds to pointwise mutual information in terms of stochastic Fisher information and show these bounds average to known results in the literature for bounds to mutual information in terms of Fisher information. These…

Communication efficient distributed mean estimation is an important primitive that arises in many distributed learning and optimization scenarios such as federated learning. Without any probabilistic assumptions on the underlying data, we…

The information-theoretic framework of Russo and J. Zou (2016) and Xu and Raginsky (2017) provides bounds on the generalization error of a learning algorithm in terms of the mutual information between the algorithm's output and the training…

Secure multi-party computation is a central problem in modern cryptography. An important sub-class of this are problems of the following form: Alice and Bob desire to produce sample(s) of a pair of jointly distributed random variables. Each…

Several proofs of the monotonicity of the non-Gaussianness (divergence with respect to a Gaussian random variable with identical second order statistics) of the sum of n independent and identically distributed (i.i.d.) random variables were…

This work proposes a novel outer bound for the Gaussian cognitive interference channel in strong interference at the primary receiver based on the capacity of a multi-antenna broadcast channel with degraded message set. It then shows that…

A family of skew information quantities is obtained, in which the well-known Wigner-Yanase skew information and quantum Fisher information stand as special cases. A transparent proof of convexity of the generalized skew information is…

Since the celebrated works of Russo and Zou (2016,2019) and Xu and Raginsky (2017), it has been well known that the generalization error of supervised learning algorithms can be bounded in terms of the mutual information between their input…

Information transmission over discrete-time channels with memoryless additive noise obeying a Cauchy, rather than Gaussian, distribution, are studied. The channel input satisfies an average power constraint. Upper and lower bounds to such…

We derive information-theoretic converses (i.e., lower bounds) for the minimum time required by any algorithm for distributed function computation over a network of point-to-point channels with finite capacity, where each node of the…

Mixtures of high dimensional Gaussian distributions have been studied extensively in statistics and learning theory. While the total variation distance appears naturally in the sample complexity of distribution learning, it is analytically…

This work discusses how to derive upper bounds for the expected generalisation error of supervised learning algorithms by means of the chaining technique. By developing a general theoretical framework, we establish a duality between…

We present a new method to propagate lower bounds on conditional probability distributions in conventional Bayesian networks. Our method guarantees to provide outer approximations of the exact lower bounds. A key advantage is that we can…

In this paper, we first provide three general norm inequalities, which are used to give new uncertainty relations of any finite observables and quantum channels via metric-adjusted skew information. The results are applicable to its special…

We study communication systems over band-limited Additive White Gaussian Noise (AWGN) channels in which the transmitter output is constrained to be symmetric binary (bi-polar). In this work we improve the original Ozarov-Wyner-Ziv (OWZ)…

Many of the classical and recent relations between information and estimation in the presence of Gaussian noise can be viewed as identities between expectations of random quantities. These include the I-MMSE relationship of Guo et al.; the…

This paper deals with arbitrarily distributed finite-power input signals observed through an additive Gaussian noise channel. It shows a new formula that connects the input-output mutual information and the minimum mean-square error (MMSE)…

Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios.…

Sensor placement approaches in networks often involve using information-theoretic measures such as entropy and mutual information. We prove that mutual information abides by submodularity and is non-decreasing when considering the mutual…