Related papers: $f$-divergence Inequalities
This paper considers derivation of $f$-divergence inequalities via the approach of functional domination. Bounds on an $f$-divergence based on one or several other $f$-divergences are introduced, dealing with pairs of probability measures…
New upper bounds on the relative entropy are derived as a function of the total variation distance. One bound refines an inequality by Verd\'{u} for general probability measures. A second bound improves the tightness of an inequality by…
A simple method is shown to provide optimal variational bounds on $f$-divergences with possible constraints on relative information extremums. Known results are refined or proved to be optimal as particular cases.
The concept of classical $f$-divergences gives a unified framework to construct and study measures of dissimilarity of probability distributions; special cases include the relative entropy and the R\'enyi divergences. Various quantum…
This work provides data-processing and majorization inequalities for $f$-divergences, and it considers some of their applications to coding problems. This work also provides tight bounds on the R\'{e}nyi entropy of a function of a discrete…
A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…
R\'enyi divergence is related to R\'enyi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon's entropy, and comes up in many settings. It was introduced by R\'enyi as…
Tight bounds for several symmetric divergence measures are introduced, given in terms of the total variation distance. Each of these bounds is attained by a pair of 2 or 3-element probability distributions. An application of these bounds…
The families of $f$-divergences (e.g. the Kullback-Leibler divergence) and Integral Probability Metrics (e.g. total variation distance or maximum mean discrepancies) are widely used to quantify the similarity between probability…
This paper is focused on $f$-divergences, consisting of three main contributions. The first one introduces integral representations of a general $f$-divergence by means of the relative information spectrum. The second part provides a new…
The data processing inequality is central to information theory and motivates the study of monotonic divergences. However, it is not clear operationally we need to consider all such divergences. We establish a simple method for Pinsker…
Pinsker-type inequalities are considered for the weighted total variation distance between probability measures in terms of the R\'enyi divergence powers. They are applied in derivation of transport-entropy inequalities under moment-type…
$f$-divergences are a general class of divergences between probability measures which include as special cases many commonly used divergences in probability, mathematical statistics and information theory such as Kullback-Leibler…
This paper focuses on parameter estimation and introduces a new method for lower bounding the Bayesian risk. The method allows for the use of virtually \emph{any} information measure, including R\'enyi's $\alpha$, $\varphi$-Divergences, and…
The relative entropy and chi-squared divergence are fundamental divergence measures in information theory and statistics. This paper is focused on a study of integral relations between the two divergences, the implications of these…
Two typical fixed-length random number generation problems in information theory are considered for general sources. One is the source resolvability problem and the other is the intrinsic randomness problem. In each of these problems, the…
A new sharp inequality featuring the differential R\'enyi entropy, the R\'enyi divergence and the R\'enyi cross-entropy of a pair of probability density functions is established. The equality is reached when one of the probability density…
Quantum f-divergences are a quantum generalization of the classical notion of f-divergences, and are a special case of Petz' quasi-entropies. Many well known distinguishability measures of quantum states are given by, or derived from,…
Divergences often play important roles for study in information science so that it is indispensable to investigate their fundamental properties. There is also a mathematical significance of such results. In this paper, we introduce some…
Divergences are quantities that measure discrepancy between two probability distributions and play an important role in various fields such as statistics and machine learning. Divergences are non-negative and are equal to zero if and only…