Related papers: Sharp Inequalities for $f$-divergences
The Chernoff information between two probability measures is a statistical divergence measuring their deviation defined as their maximally skewed Bhattacharyya distance. Although the Chernoff information was originally introduced for…
The likelihood function is a fundamental component in Bayesian statistics. However, evaluating the likelihood of an observation is computationally intractable in many applications. In this paper, we propose a non-parametric approximation of…
Variational representations of $f$-divergences are central to many machine learning algorithms, with Lipschitz constrained variants recently gaining attention. Inspired by this, we define the Moreau-Yosida approximation of $f$-divergences…
Fairness-aware learning is a novel framework for classification tasks. Like regular empirical risk minimization (ERM), it aims to learn a classifier with a low error rate, and at the same time, for the predictions of the classifier to be…
Kullback-Leibler (KL) divergence is one of the most important divergence measures between probability distributions. In this paper, we prove several properties of KL divergence between multivariate Gaussian distributions. First, for any two…
Nonnegative matrix factorization (NMF) is a standard linear dimensionality reduction technique for nonnegative data sets. In order to measure the discrepancy between the input data and the low-rank approximation, the Kullback-Leibler (KL)…
In this paper, we will use a suitable tranform to investigate the sharp constants and optimizers for the following Caffarelli-Kohn-Nirenberg inequalities for a wide range of parameters $(r,p,q,s,\mu,\sigma)$ and $0\leq a\leq1$:…
Fairness has become a crucial aspect in the development of trustworthy machine learning algorithms. Current fairness metrics to measure the violation of demographic parity have the following drawbacks: (i) the average difference of model…
$\alpha$-posteriors and their variational approximations distort standard posterior inference by downweighting the likelihood and introducing variational approximation errors. We show that such distortions, if tuned appropriately, reduce…
Consider a geometric range space $(X,\c{A})$ where each data point $x \in X$ has two or more values (say $r(x)$ and $b(x)$). Also consider a function $\Phi(A)$ defined on any subset $A \in (X,\c{A})$ on the sum of values in that range e.g.,…
We study the relation between the total variation (TV) and Hellinger distances between two Gaussian location mixtures. Our first result establishes a general upper bound: for any two mixing distributions supported on a compact set, the…
In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two…
The inverse tangent function can be bounded by different inequalities, for example by Shafer's inequality. In this publication, we propose a new sharp double inequality, consisting of a lower and an upper bound, for the inverse tangent…
The (global) Lipschitz smoothness condition is crucial in establishing the convergence theory for most optimization methods. Unfortunately, most machine learning and signal processing problems are not Lipschitz smooth. This motivates us to…
Importance sampling approximates expectations with respect to a target measure by using samples from a proposal measure. The performance of the method over large classes of test functions depends heavily on the closeness between both…
The fractional Feynman-Kac equations describe the distribution of functionals of non-Brownian motion, or anomalous diffusion, including two types called the forward and backward fractional Feynman-Kac equations, where the fractional…
Empirical phi-divergence test-statistics have demostrated to be a useful technique for the simple null hypothesis to improve the finite sample behaviour of the classical likelihood ratio test-statistic, as well asfor model misspecification…
In this paper we study direct and inverse approximation inequalities in $L^{p}(\mathbb{R}^{d})$, $1<p<\infty$, with the Dunkl weight. We obtain these estimates in their sharp form substantially improving previous results. We also establish…
The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields…
Kullback--Leibler (KL) divergence is a fundamental measure of the dissimilarity between two probability distributions, but it can become unstable in high-dimensional settings due to its sensitivity to mismatches in distributional support.…