Related papers: Distances between probability distributions of dif…
We study different ways of determining the mean distance $ < r_n >$ between a reference point and its $n$-th neighbour among random points distributed with uniform density in a $D$-dimensional Euclidean space. First we present a heuristic…
Embedding probability distributions into reproducing kernel Hilbert spaces (RKHS) has enabled powerful nonparametric methods such as the maximum mean discrepancy (MMD), a statistical distance with strong theoretical and computational…
In this paper, we derive some upper and lower bounds and inequalities for the total variation distance (TVD) and the Kullback-Leibler divergence (KLD), also known as the relative entropy, between two probability measures $\mu$ and $\nu$…
The quantale of distance distributions is of fundamental importance for understanding probabilistic metric spaces as enriched categories. Motivated by the categorical interpretation of partial metric spaces, we are led to investigate the…
There are many applications that benefit from computing the exact divergence between 2 discrete probability measures, including machine learning. Unfortunately, in the absence of any assumptions on the structure or independencies within…
In the study of dynamical and physical systems, the input parameters are often uncertain or randomly distributed according to a measure $\varrho$. The system's response $f$ pushes forward $\varrho$ to a new measure $f\circ \varrho$ which we…
The question of optimally approximating an arbitrary probability measure in the Wasserstein distance by a discrete one with uniform weights is considered. Estimates are obtained for the optimal approximation distance, with an explicit rate…
Complex, high-dimensional data is ubiquitous across many scientific disciplines, including machine learning, biology, and the social sciences. One of the primary methods of visualizing these datasets is with two-dimensional scatter plots…
In this paper, we address the classification of instances each characterized not by a singular point, but by a distribution on a vector space. We employ the Wasserstein metric to measure distances between distributions, which are then used…
The Jensen-Shannon divergence is a renown bounded symmetrization of the unbounded Kullback-Leibler divergence which measures the total Kullback-Leibler divergence to the average mixture distribution. However the Jensen-Shannon divergence…
Consider the nonparametric logistic regression problem. In the logistic regression, we usually consider the maximum likelihood estimator, and the excess risk is the expectation of the Kullback-Leibler (KL) divergence between the true and…
If the probability distribution model aims to approximate the hidden mother distribution, it is imperative to establish a useful criterion for the resemblance between the mother and the model distributions. This study proposes a criterion…
A loss function measures the discrepancy between the true values and their estimated fits, for a given instance of data. In classification problems, a loss function is said to be proper if a minimizer of the expected loss is the true…
We are interested in the estimation of the distance in total variation $$ \Delta := \|P_{f(X)} - P_{g(X)}\|_{\mathrm var} $$ between distributions of random variables $f(X)$ and $g(X)$ in terms of proximity of $f$ and $g.$ We propose a…
This paper deals with maximization of classical $f$-divergence between the distributions of a measurement outputs of a given pair of quantum states. $f$-divergence $D_{f}$ between the probability density functions $p_{1}$ and $p_{2}$ over a…
There are many information and divergence measures exist in the literature on information theory and statistics. The most famous among them are Kullback-Leiber's (1951)relative information and Jeffreys (1946) J-divergence, Information…
Variational Inference approximates an unnormalized distribution via the minimization of Kullback-Leibler (KL) divergence. Although this divergence is efficient for computation and has been widely used in applications, it suffers from some…
Applications in data science, shape analysis and object classification frequently require comparison of probability distributions defined on different ambient spaces. To accomplish this, one requires a notion of distance on a given class of…
The Wasserstein distance is a distance between two probability distributions and has recently gained increasing popularity in statistics and machine learning, owing to its attractive properties. One important approach to extending this…
Two geometrical structures have been extensively studied for a manifold of probability distributions. One is based on the Fisher information metric, which is invariant under reversible transformations of random variables, while the other is…