Related papers: Nested Inequalities Among Divergence Measures
We discuss different statistical distances in probability space, with emphasis on the Jensen-Shannon divergence, vis-a-vis {\it metrics} in Hilbert space and their relationship with Fisher's information measure. This study provides further…
In this paper we provide the asymptotic theory of the general of $\phi$-divergences measures, which includes the most common divergence measures : Renyi and Tsallis families and the Kullback-Leibler measure. Instead of using the Parzen…
In many applications in biology, engineering and economics, identifying similarities and differences between distributions of data from complex processes requires comparing finite categorical samples of discrete counts. Statistical…
We study a family of inequalities on pairs of measure spaces involving functions defined on product domains. Our main result establishes a Jensen-type inequality under a general product-measure framework, extending classical inequalities…
Complex, high-dimensional data is ubiquitous across many scientific disciplines, including machine learning, biology, and the social sciences. One of the primary methods of visualizing these datasets is with two-dimensional scatter plots…
In previous work the authors defined the k-th order simplicial distance between probability distributions which arises naturally from a measure of dispersion based on the squared volume of random simplices of dimension k. This theory is…
Four problems related to information divergence measures defined on finite alphabets are considered. In three of the cases we consider, we illustrate a contrast which arises between the binary-alphabet and larger-alphabet settings. This is…
This is the second of two papers wherein we estimate multiscale least squares approximations of certain measures by Menger-type curvatures. More specifically, we study an arbitrary d-regular measure on a real separable Hilbert space. The…
Comparing probability distributions is an indispensable and ubiquitous task in machine learning and statistics. The most common way to compare a pair of Borel probability measures is to compute a metric between them, and by far the most…
We consider a linear ill-posed equation in the Hilbert space setting. Multiple independent unbiased measurements of the right hand side are available. A natural approach is to take the average of the measurements as an approximation of the…
We prove a refinement of the inequality by Hoffmann-Jorgensen that is significant for three reasons. First, our result improves on the state-of-the-art even for real-valued random variables. Second, the result unifies several versions in…
This paper deals with three major types of convergence of probability measures on metric spaces: weak convergence, setwise converges, and convergence in the total variation. First, it describes and compares necessary and sufficient…
The forward Kullback-Leibler (KL) divergence is a ubiquitous objective for fitting a parameterized distribution to samples due to its tractability and equivalence to maximum likelihood estimation (MLE). Its inherent asymmetry, however, may…
Automated feature selection is important for text categorization to reduce the feature size and to speed up the learning process of classifiers. In this paper, we present a novel and efficient feature selection framework based on the…
Knowing if a model will generalize to data 'in the wild' is crucial for safe deployment. To this end, we study model disagreement notions that consider the full predictive distribution - specifically disagreement based on Hellinger…
In the present paper we prove several results concerning the existence of low-discrepancy point sets with respect to an arbitrary non-uniform measure $\mu$ on the $d$-dimensional unit cube. We improve a theorem of Beck, by showing that for…
Parametric adversarial divergences, which are a generalization of the losses used to train generative adversarial networks (GANs), have often been described as being approximations of their nonparametric counterparts, such as the…
We consider the variance of a function of $n$ independent random variables and provide new inequalities which, in particular, extend previous results obtained for symmetric functions in the i.i.d.~setting. For instance, we obtain various…
Convex analysis is fundamental to proving inequalities that have a wide variety of applications in economics and mathematics. In this paper we provide Jensen-type inequalities for functions that are, intuitively, "very" convex. These…
We study the time evolution of four distance measures in the presence of initial systemenvironment correlations. It is well-known that the trace distance between two quantum states of an open system may increase due to initial correlations…