相关论文: Linear Bounds between Contraction Coefficients for…

In this paper, tight upper and lower bounds are derived on the weighted sum of minimum mean-squared errors for additive Gaussian noise channels. The bounds are obtained by constraining the input distribution to be close to a Gaussian…

The distortion-rate function of output-constrained lossy source coding with limited common randomness is analyzed for the special case of squared error distortion measure. An explicit expression is obtained when both source and…

In this paper, we show that although the minimizers of cross-entropy and related classification losses are off at infinity, network weights learned by gradient flow converge in direction, with an immediate corollary that network…

This paper investigates the joint source-channel coding problem of sending two correlated memoryless sources with common part over a memoryless multiple access channel (MAC). An inner bound and two outer bounds on the achievable distortion…

The two-user broadcast channel (BC) with receivers connected by bidirectional cooperation links of finite capacities, known as conferencing decoders, is considered. A novel capacity region outer bound is established based on multiple…

Smooth Csisz\'ar $f$-divergences can be expressed as integrals over so-called hockey stick divergences. This motivates a natural quantum generalization in terms of quantum Hockey stick divergences, which we explore here. Using this recipe,…

We show that approximating the trace norm contraction coefficient of a quantum channel within a constant factor is NP-hard. Equivalently, this shows that determining the optimal success probability for encoding a bit in a quantum system…

In their seminal work, Bennett et al. [IEEE Trans. Inf. Theory (2002)] showed that, with sufficient shared randomness, one noisy channel can simulate another at a rate equal to the ratio of their capacities. We establish that when coding…

In this paper we provide a new inequality useful for the proofs of strong converse theorems in the multiterminal information theory. We apply this inequality to the recent work by Tyagi and Watanabe on the strong converse theorem for the…

This paper shows new general nonasymptotic achievability and converse bounds and performs their dispersion analysis for the lossy compression problem in which the compressor observes the source through a noisy channel. While this problem is…

Information divergences allow one to assess how close two distributions are from each other. Among the large panel of available measures, a special attention has been paid to convex $\varphi$-divergences, such as Kullback-Leibler,…

We derive a few extended versions of the Kraft inequality for lossy compression, which pave the way to the derivation of several refinements and extensions of the well known Shannon lower bound in a variety of instances of rate-distortion…

Communication bottlenecks and the presence of stragglers pose significant challenges in distributed learning (DL). To deal with these challenges, recent advances leverage unbiased compression functions and gradient coding. However, the…

Tight bounds for several symmetric divergence measures are derived in terms of the total variation distance. It is shown that each of these bounds is attained by a pair of 2 or 3-element probability distributions. An application of these…

We investigate the optimal performance of dense sensor networks by studying the joint source-channel coding problem. The overall goal of the sensor network is to take measurements from an underlying random process, code and transmit those…

We study the rate-distortion problem for both scalar and vector memoryless heavy-tailed $\alpha$-stable sources ($0 < \alpha < 2$). Using a recently defined notion of ``strength" as a power measure, we derive the rate-distortion function…

Exponential families are statistical models which are the workhorses in statistics, information theory, and machine learning among others. An exponential family can either be normalized subtractively by its cumulant or free energy function…

We consider the problem of estimating probability density functions based on sample data, using a finite mixture of densities from some component class. To this end, we introduce the $h$-lifted Kullback--Leibler (KL) divergence as a…

In the loss function of Variational Autoencoders there is a well known tension between two components: the reconstruction loss, improving the quality of the resulting images, and the Kullback-Leibler divergence, acting as a regularizer of…

The performance of machine learning classification algorithms are evaluated by estimating metrics, often from the confusion matrix, using training data and cross-validation. However, these do not prove that the best possible performance has…