Related papers: Distances between probability distributions of dif…
Generalization error bounds are essential to understanding machine learning algorithms. This paper presents novel expected generalization error upper bounds based on the average joint distribution between the output hypothesis and each…
The sliced Wasserstein distance as well as its variants have been widely considered in comparing probability measures defined on $\mathbb R^d$. Here we derive the notion of sliced Wasserstein distance for measures on an infinite dimensional…
$f$-divergences are a general class of divergences between probability measures which include as special cases many commonly used divergences in probability, mathematical statistics and information theory such as Kullback-Leibler…
In this paper we introduce the intuitive notion of trivergence of probability distributions (TPD). This notion allow us to calculate the similarity among triplets of objects. For this computation, we can use the well known measures of…
The Wasserstein distance provides a notion of dissimilarities between probability measures, which has recent applications in learning of structured data with varying size such as images and text documents. In this work, we study the…
This paper deals with the problem of quantifying the approximation a probability measure by means of an empirical (in a wide sense) random probability measure, depending on the first n terms of a sequence of random elements. In Section 2,…
There are many information and divergence measures exist in the literature on information theory and statistics. The most famous among them are Kullback-Leibler (1951) relative information and Jeffreys (1951) J-divergence. Sibson (1969)…
We discuss two ways to construct standard probability measures, called push-down measures, from internal probability measures. We show that the Wasserstein distance between an internal probability measure and its push-down measure is…
We show that computing the total variation distance between two product distributions is $\#\mathsf{P}$-complete. This is in stark contrast with other distance measures such as Kullback-Leibler, Chi-square, and Hellinger, which tensorize…
The Kullback-Leibler (KL) divergence is a fundamental equation of information theory that quantifies the proximity of two probability distributions. Although difficult to understand by examining the equation, an intuition and understanding…
The curse of dimensionality is a common phenomenon which affects analysis of datasets characterized by large numbers of variables associated with each point. Problematic scenarios of this type frequently arise in classification algorithms…
In this paper, we study the statistical and geometrical properties of the Kullback-Leibler divergence with kernel covariance operators (KKL) introduced by Bach [2022]. Unlike the classical Kullback-Leibler (KL) divergence that involves…
Density-based directed distances -- particularly known as divergences -- between probability distributions are widely used in statistics as well as in the adjacent research fields of information theory, artificial intelligence and machine…
Starting with a similarity function between objects, it is possible to define a distance metric on pairs of objects, and more generally on probability distributions over them. These distance metrics have a deep basis in functional analysis,…
The Wasserstein metric is an important measure of distance between probability distributions, with applications in machine learning, statistics, probability theory, and data analysis. This paper provides upper and lower bounds on…
This paper deals with a new Bayesian approach to the standard one-sample $z$- and $t$- tests. More specifically, let $x_1,\ldots,x_n$ be an independent random sample from a normal distribution with mean $\mu$ and variance $\sigma^2$. The…
We establish conditions to characterize probability measures by their $L^{p}$-quantization error functions in both $\mathbb{R}^{d}$ and Hilbert settings. This characterization is two-fold: static (identity of two distributions) and dynamic…
We present a PAC-Bayes-style generalization bound which enables the replacement of the KL-divergence with a variety of Integral Probability Metrics (IPM). We provide instances of this bound with the IPM being the total variation metric and…
Collections of probability distributions arise in a variety of applications ranging from user activity pattern analysis to brain connectomics. In practice these distributions can be defined over diverse domain types including finite…
Wasserstein distances are metrics on probability distributions inspired by the problem of optimal mass transportation. Roughly speaking, they measure the minimal effort required to reconfigure the probability mass of one distribution in…