Related papers: R\'enyi Divergence and Kullback-Leibler Divergence

R\'enyi divergence is related to R\'enyi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as…

We give an overview of various results and methods related to information-theoretic distances of R\'enyi type in the light of their applications to the central limit theorem (CLT). The first part (Sections 1-9) is devoted to the total…

In this work we introduce a family of transformations, named \textit{divergence transformations}, interpolating between any pair of probability density functions sharing the same support. We prove the remarkable property that the whole…

The redundancy for universal lossless compression of discrete memoryless sources in Campbell's setting is characterized as a minimax R\'enyi divergence, which is shown to be equal to the maximal $\alpha$-mutual information via a generalized…

In the problem of binary quantum channel discrimination with product inputs, the supremum of all type II error exponents for which the optimal type I errors go to zero is equal to the Umegaki channel relative entropy, while the infimum of…

Fawzi and Fawzi recently defined the sharp R\'enyi divergence, $D_\alpha^\#$, for $\alpha \in (1, \infty)$, as an additional quantum R\'enyi divergence with nice mathematical properties and applications in quantum channel discrimination and…

We derive a new variational formula for the R\'enyi family of divergences, $R_\alpha(Q\|P)$, between probability measures $Q$ and $P$. Our result generalizes the classical Donsker-Varadhan variational formula for the Kullback-Leibler…

We investigate quantum R\'enyi entropic quantities, specifically those derived from 'sandwiched' divergence. This divergence is one of several proposed R\'enyi generalisations of the quantum relative entropy. We may define R\'enyi…

This work explores properties of Strong Data-Processing constants for R\'enyi Divergences. Parallels are made with the well-studied $\varphi$-Divergences, and it is shown that the order $\alpha$ of R\'enyi Divergences dictates whether…

Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These…

This work presents an infinite-dimensional generalization of the correspondence between the Kullback-Leibler and R\'enyi divergences between Gaussian measures on Euclidean space and the Alpha Log-Determinant divergences between symmetric,…

A well-known technique in estimating probabilities of rare events in general and in information theory in particular (used, e.g., in the sphere-packing bound), is that of finding a reference probability measure under which the event of…

This paper extends some geometric properties of a one-parameter family of relative entropies. These arise as redundancies when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative…

We propose R\'enyi information generating function and discuss its properties. A connection between the R\'enyi information generating function and the diversity index is proposed for discrete type random variables. The relation between the…

The concept of classical $f$-divergences gives a unified framework to construct and study measures of dissimilarity of probability distributions; special cases include the relative entropy and the R\'enyi divergences. Various quantum…

We obtain formulas for Petz-R\'enyi and Umegaki relative entropy from the idea of distribution of a positive selfadjoint operator. Classical results on R\'enyi and Kullback-Leibler divergences are applied to obtain new results and new…

We introduce a new quantum R\'enyi divergence $D^{\#}_{\alpha}$ for $\alpha \in (1,\infty)$ defined in terms of a convex optimization program. This divergence has several desirable computational and operational properties such as an…

The analogues of Arimoto's definition of conditional R\'enyi entropy and R\'enyi mutual information are explored for abstract alphabets. These quantities, although dependent on the reference measure, have some useful properties similar to…

Divergences are quantities that measure discrepancy between two probability distributions and play an important role in various fields such as statistics and machine learning. Divergences are non-negative and are equal to zero if and only…

In this paper, we examine the existence of the R\'enyi divergence between two time invariant general hidden Markov models with arbitrary positive initial distributions. By making use of a Markov chain representation of the probability…