Related papers: A Tight Upper Bound on Mutual Information
We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…
In this paper, tight upper and lower bounds are derived on the weighted sum of minimum mean-squared errors for additive Gaussian noise channels. The bounds are obtained by constraining the input distribution to be close to a Gaussian…
The single-letter characterisation of the entanglement-assisted capacity of a quantum channel is one of the seminal results of quantum information theory. In this paper, we consider a modified communication scenario in which the receiver is…
Statistical divergences are important tools in data analysis, information theory, and statistical physics, and there exist well known inequalities on their bounds. However, in many circumstances involving temporal evolution, one needs…
We present novel data-processing inequalities relating the mutual information and the directed information in systems with feedback. The internal blocks within such systems are restricted only to be causal mappings, but are allowed to be…
In this paper, we consider a scenario where a source node wishes to broadcast two confidential messages for two respective receivers, while a wire-tapper also receives the transmitted signal. This model is motivated by wireless…
The existing upper and lower bounds between entropy and error probability are mostly derived from the inequality of the entropy relations, which could introduce approximations into the analysis. We derive analytical bounds based on the…
Detailed derivations of two bounds of the minimum mean-square error (MMSE) of complex-valued multiple-input multiple-output (MIMO) systems are proposed for performance evaluation. Particularly, the lower bound is derived based on a…
We derive various error exponents in the bee identification problem under two different decoding rules. Under na\"ive decoding, which decodes each bee independently of the others, we analyze a general discrete memoryless channel and a…
We derive upper and lower bounds on the reliability function for the common-message discrete memoryless broadcast channel with variable-length feedback. We show that the bounds are tight when the broadcast channel is stochastically…
We consider the discrete memoryless degraded broadcast channels with feedback. We prove that the error probability of decoding tends to one exponentially for rates outside the capacity region and derive an explicit lower bound of this…
The performance of maximum-likelihood (ML) decoded binary linear block codes is addressed via the derivation of tightened upper bounds on their decoding error probability. The upper bounds on the block and bit error probabilities are valid…
The mutual information (MI) between two random variables is an important correlation measure in data analysis. The Shannon entropy of a joint probability distribution is the variable part under fixed marginals. We aim to minimize and…
Given two channels that convey information about the same random variable, we introduce two measures of the unique information of one channel with respect to the other. The two quantities are based on the notion of generalized weighted Le…
We derive an exponentially decaying upper-bound on the unnormalized amount of information leaked to the wire-tapper in Wyner's wire-tap channel setting. We characterize the exponent of the bound as a function of the randomness used by the…
This paper investigates achievable information rates and error exponents of mismatched decoding when the channel belongs to the class of channels that are close to the decoding metric in terms of relative entropy. For both discrete- and…
We provide lower and upper bounds on the information transmission capacity of one single use of a classical-quantum channel. The lower bound is expressed in terms of the Hoeffding capacity, that we define similarly to the Holevo capacity,…
We show how to use properties of the quantum conditional mutual information to obtain continuity bounds for information characteristics of quantum channels depending on their input dimension. First we prove tight estimates for variation of…
Due to the short and bursty incoming messages, channel access activities in a wireless random access system are often fractional. The lack of frequent data support consequently makes it difficult for the receiver to estimate and track the…
Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging. To establish tractable and scalable objectives, recent work has turned to variational…