Related papers: Sharp Inequalities for $f$-divergences
The aim of this paper is to introduce new statistical criterions for estimation, suitable for inference in models with common continuous support. This proposal is in the direct line of a renewed interest for divergence based inference tools…
A class of distance measures on probabilities -- the integral probability metrics (IPMs) -- is addressed: these include the Wasserstein distance, Dudley metric, and Maximum Mean Discrepancy. IPMs have thus far mostly been used in more…
In this paper, the Orlicz addition of measures is proposed and an interpretation of the $f$-divergence is provided based on a linear Orlicz addition of two measures. Fundamental inequalities, such as, a dual functional…
We study the convergence of the new family of mimetic finite difference schemes for linear diffusion problems recently proposed in [38]. In contrast to the conventional approach, the diffusion coefficient enters both the primary mimetic…
Adequacy for estimation between an inferential method and a model can be de{\ldots}ned through two main requirements: {\ldots}rstly the inferential tool should de{\ldots}ne a well posed problem when applied to the model; secondly the…
This paper considers derivation of $f$-divergence inequalities via the approach of functional domination. Bounds on an $f$-divergence based on one or several other $f$-divergences are introduced, dealing with pairs of probability measures…
Consider the nonparametric logistic regression problem. In the logistic regression, we usually consider the maximum likelihood estimator, and the excess risk is the expectation of the Kullback-Leibler (KL) divergence between the true and…
This work presents an upper-bound to value that the Kullback-Leibler (KL) divergence can reach for a class of probability distributions called quantum distributions (QD). The aim is to find a distribution $U$ which maximizes the KL…
The problem of f-divergence estimation is important in the fields of machine learning, information theory, and statistics. While several nonparametric divergence estimators exist, relatively few have known convergence properties. In…
We review recent results about the maximal values of the Kullback-Leibler information divergence from statistical models defined by neural networks, including naive Bayes models, restricted Boltzmann machines, deep belief networks, and…
In this paper we have considered a single inequality having 11 known divergence measures. This inequality include measures like: Jeffryes-Kullback-Leiber J-divergence, Jensen-Shannon divergence (Burbea-Rao, 1982), arithmetic-geometric mean…
We consider the problem of allocating samples to a finite set of discrete distributions in order to learn them uniformly well in terms of four common distance measures: $\ell_2^2$, $\ell_1$, $f$-divergence, and separation distance. To…
It is shown that the $f$-divergence between two probability measures $P$ and $R$ equals the supremum of the same $f$-divergence computed over all finite measurable partitions of the original space, thus generalizing results previously…
Hypothesis testing in high dimensional data is a notoriously difficult problem without direct access to competing models' likelihood functions. This paper argues that statistical divergences can be used to quantify the difference between…
We consider the fundamental problem of estimating a discrete distribution on a domain of size $K$ with high probability in Kullback-Leibler divergence. We provide upper and lower bounds on the minimax estimation rate, which show that the…
A loss function measures the discrepancy between the true values and their estimated fits, for a given instance of data. In classification problems, a loss function is said to be proper if a minimizer of the expected loss is the true…
We design accelerated algorithms with improved rates for several fundamental classes of optimization problems. Our algorithms all build upon techniques related to the analysis of primal-dual extragradient methods via relative Lipschitzness…
In this paper we consider one parameter generalizations of some non - symmetric divergence measures. Measures are \textit{relative information}, $\chi ^2 - $\textit{divergence}, \textit{relative J-divergence}, \textit{relative…
The aim of this paper is to investigate extremum problems with pay-off being the total variational distance metric defined on the space of probability measures, subject to linear functional constraints on the space of probability measures,…
We introduce a new class of Frank-Wolfe algorithms for minimizing differentiable functionals over probability measures. This framework can be shown to encompass a diverse range of tasks in areas such as artificial intelligence,…