Related papers: $f$-divergence Inequalities
The relative log-concavity ordering $\leq_{\mathrm{lc}}$ between probability mass functions (pmf's) on non-negative integers is studied. Given three pmf's $f,g,h$ that satisfy $f\leq_{\mathrm{lc}}g\leq_{\mathrm{lc}}h$, we present a pair of…
We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of $f$-Divergences. The results are then applied to specific settings of interest and compared to other…
Atar, Chowdhary and Dupuis have recently exhibited a variational formula for exponential integrals of bounded measurable functions in terms of R\'enyi divergences. We develop a variational characterization of the R\'enyi divergences between…
This paper proposes a unifying variational approach for proving and extending some fundamental information theoretic inequalities. Fundamental information theory results such as maximization of differential entropy, minimization of Fisher…
Many problems in machine learning can be formulated as optimizing a convex functional over a vector space of measures. This paper studies the convergence of the mirror descent algorithm in this infinite-dimensional setting. Defining Bregman…
This paper introduces a comprehensive framework for complex-valued probability measures and explores their novel applications in information theory and statistical analysis. We define a complex probability measure as a phase-modulated…
We present a unified approach to get explicit formulas for utility maximising strategies in Exponential Levy models. This approach is related to $f$-divergence minimal martingale measures and based on a new concept of preservation of the…
Interactive statistical decision making (ISDM) features algorithm-dependent data generated through interaction. Existing information-theoretic lower bounds in ISDM largely target expected risk, while tail-sensitive objectives are less…
In this paper, we propose the Fourier Discrepancy Function, a new discrepancy to compare discrete probability measures. We show that this discrepancy takes into account the geometry of the underlying space. We prove that the Fourier…
We review old and recent finite de Finetti theorems in total variation distance and in relative entropy, and we highlight their connections with bounds on the difference between sampling with and without replacement. We also establish two…
Quantum information processing is limited, in practice, to efficiently implementable operations. This motivates the study of quantum divergences that preserve their operational meaning while faithfully capturing these computational…
We make a systematic study of standard $f$-divergences in general von Neumann algebras. An important ingredient of our study is to extend Kosaki's variational expression of the relative entropy to an arbitary standard $f$-divergence, from…
We study several constrained variational problem in the 2-Wasserstein metric for which the set of probability densities satisfying the constraint is not closed. For example, given a probability density $F_0$ on $\R^d$ and a time-step $h>0$,…
This work belongs to the framework of inverse problems with linear model. The resolution of this type of problem consists in minimizing (possibly under constraints) a function of discrepancy between the measurements and a physical model of…
In this paper we consider one parameter generalizations of some non - symmetric divergence measures. Measures are \textit{relative information}, $\chi ^2 - $\textit{divergence}, \textit{relative J-divergence}, \textit{relative…
Motivated by problems in contact mechanics, we propose a duality approach for computing approximations and associated a posteriori error bounds to solutions of variational inequalities of the first kind. The proposed approach improves upon…
The article presents new sup-sums principles for integral F-divergence for arbitrary convex function F and arbitrary (not necessarily positive and absolutely continuous) measures. As applications of these results we derive the corresponding…
This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal…
We propose a general approach for quantitative convergence analysis of non-reversible Markov processes, based on the concept of second-order lifts and a variational approach to hypocoercivity. To this end, we introduce the flow Poincar{\'e}…
We investigate the contraction coefficients derived from strong data processing inequalities for the $E_\gamma$-divergence. By generalizing the celebrated Dobrushin's coefficient from total variation distance to $E_\gamma$-divergence, we…