Related papers: Sharp Inequalities for $f$-divergences
The book is structured into four main chapters. Chapter 1 introduces the foundational concepts of divergence measures, including the well-known Kullback-Leibler divergence and its limitations. It then presents a detailed exploration of…
Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These…
In the framework of a real Hilbert space we consider the problem of approaching solutions to a class of hierarchical variational inequality problems, subsuming several other problem classes including certain mathematical programs under…
This paper is an attempt to set a justification for making use of some dicrepancy indexes, starting from the classical Maximum Likelihood definition, and adapting the corresponding basic principle of inference to situations where…
Probabilistic models are often trained by maximum likelihood, which corresponds to minimizing a specific f-divergence between the model and data distribution. In light of recent successes in training Generative Adversarial Networks,…
For a probability measure on a real separable Hilbert space, we are interested in "volume-based" approximations of the d-dimensional least squares error of it, i.e., least squares error with respect to a best fit d-dimensional affine…
In this paper, the concept of the classical $f$-divergence (for a pair of measures) is extended to the mixed $f$-divergence (for multiple pairs of measures). The mixed $f$-divergence provides a way to measure the difference between multiple…
This paper investigates the impact of distributional uncertainty on key risk measures under the partial knowledge of underlying distributions characterized by their first two moments and shape information (specifically symmetry and/or…
Divergence functions are interesting discrepancy measures. Even though they are not true distances, we can use them to measure how separated two points are. Curiously enough, when they are applied to random variables, they lead to a notion…
For decreasing sequences $\{t_{n}\}_{n=1}^{\infty}$ converging to zero, we obtain the almost everywhere convergence results for sequences of Schr\"{o}dinger means $e^{it_{n}\Delta}f$, where $f \in H^{s}(\mathbb{R}^{N}), N\geq 2$. The…
We generalise the classical Pinsker inequality which relates variational divergence to Kullback-Liebler divergence in two ways: we consider arbitrary f-divergences in place of KL divergence, and we assume knowledge of a sequence of values…
Knowing if a model will generalize to data 'in the wild' is crucial for safe deployment. To this end, we study model disagreement notions that consider the full predictive distribution - specifically disagreement based on Hellinger…
This paper is focused on $f$-divergences, consisting of three main contributions. The first one introduces integral representations of a general $f$-divergence by means of the relative information spectrum. The second part provides a new…
Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic…
In this paper, we introduce new classes of divergences by extending the definitions of the Bregman divergence and the skew Jensen divergence. These new divergence classes (g-Bregman divergence and skew g-Jensen divergence) satisfy some…
We show that the $f$-divergence between any two densities of potentially different location-scale families can be reduced to the calculation of the $f$-divergence between one standard density with another location-scale density. It follows…
Message identification (M-I) divergence is an important measure of the information distance between probability distributions, similar to Kullback-Leibler (K-L) and Renyi divergence. In fact, M-I divergence with a variable parameter can…
For some smooth special case of generalized $\varphi-$divergences as well as of new divergences (called scaled shift divergences), we derive approximations of the omnipresent (weighted) $\ell_{1}-$distance and (weighted) $\ell_{1}-$norm.
Pinsker's inequality sets a lower bound on the Umegaki divergence of two quantum states in terms of their trace distance. In this work, we formulate corresponding estimates for a variety of quantum and classical divergences including…
Good robust estimators can be tuned to combine a high breakdown point and a specified asymptotic efficiency at a central model. This happens in regression with MM- and tau-estimators among others. However, the finite-sample efficiency of…