Related papers: R\'enyi Divergence and Kullback-Leibler Divergence
We establish quantitative comparisons between classical distances for probability distributions belonging to the class of convex probability measures. Distances include total variation distance, Wasserstein distance, Kullback-Leibler…
We present a variational characterization for the R\'{e}nyi divergence of order infinity. Our characterization is related to guessing: the objective functional is a ratio of maximal expected values of a gain function applied to the…
The entropy of probability distribution defined by Shannon has several extensions. R\'enyi entropy is one of the general extensions of Shannon entropy and is widely used in engineering, physics, and so on. On the other hand, the quantum…
Regularized optimal transport (OT) has received much attention in recent years starting from Cuturi's introduction of Kullback-Leibler (KL) divergence regularized OT. In this paper, we propose regularizing the OT problem using the family of…
Since their introduction in the early sixties, the R\'enyi entropies have been used in many contexts, ranging from information theory to astrophysics, turbulence phenomena and others. In this note, we enlighten the main connections between…
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…
This work presents an upper-bound to value that the Kullback-Leibler (KL) divergence can reach for a class of probability distributions called quantum distributions (QD). The aim is to find a distribution $U$ which maximizes the KL…
Strong data processing inequalities (SDPI) are an important object of study in Information Theory and have been well studied for $f$-divergences. Universal upper and lower bounds have been provided along with several applications,…
Disagreement is an essential element of science and life in general. The language of probabilities and statistics is often used to describe disagreements quantitatively. In practice, however, we want much more than that. We want…
The aim of this work is to provide bounds connecting two probability measures of the same event using R\'enyi $\alpha$-Divergences and Sibson's $\alpha$-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence…
Estimating divergences in a consistent way is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there has been no finite sample exponential…
The aim of this paper is to study certain properties of the Kullback-Leibler distance between two positive integer numbers or between two ideals. We present some results related the entropy of a positive integer number and the divergence of…
Convergence properties of Shannon Entropy are studied. In the differential setting, it is shown that weak convergence of probability measures, or convergence in distribution, is not enough for convergence of the associated differential…
Bounds on information combining are entropic inequalities that determine how the information, or entropy, of a set of random variables can change when they are combined in certain prescribed ways. Such bounds play an important role in…
Renyi's information measures ---the Renyi information, mean, capacity, radius, and center--- are analyzed relying on the elementary properties of the Renyi divergence and the power means. The van Erven-Harremoes conjecture is proved for any…
Kullback-Leibler relative-entropy, in cases involving distributions resulting from relative-entropy minimization, has a celebrated property reminiscent of squared Euclidean distance: it satisfies an analogue of the Pythagoras' theorem. And…
We investigate bounds in the transmission of classical information through quantum systems. Our focus lies in the generalized Holevo theorem, which provides a single-letter Holevo-like inequality from arbitrary quantum distance measures.…
This paper provides tight bounds on the R\'enyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one-to-one. To that end, a tight lower bound on the R\'enyi…
We consider a two-parameter family of R\'enyi relative entropies $D_{\alpha,z}(\rho||\sigma)$ that are quantum generalisations of the classical R\'enyi divergence $D_{\alpha}(p||q)$. This family includes many known relative entropies (or…
We give systematic ways of defining monotone quantum relative entropies and (multi-variate) quantum R\'enyi divergences starting from a set of monotone quantum relative entropies. Despite its central importance in information theory, only…