Related papers: Total Variation Distance for Product Distributions…
The Kullback-Leibler divergence, the Kullback-Leibler variation, and the Bernstein "norm" are used to quantify discrepancies among probability distributions in likelihood models such as nonparametric maximum likelihood and nonparametric…
The Hellinger distance between quantum states is a significant measure in quantum information theory, known for its Riemannian and monotonic properties. It is also easier to compute than the Bures distance, another measure that shares these…
We provide optimal lower and upper bounds for the augmented Kullback-Leibler divergence in terms of the augmented total variation distance between two probability measures defined on two Euclidean spaces having different dimensions. We call…
Given samples from an unknown distribution $p$ and a description of a distribution $q$, are $p$ and $q$ close or far? This question of "identity testing" has received significant attention in the case of testing whether $p$ and $q$ are…
We study the problem of designing mechanisms when agents' valuation functions are drawn from unknown and correlated prior distributions. In particular, we are given a prior distribution $\D$, and we are interested in designing a (truthful)…
We provide an implementation to compute the flat metric in any dimension. The flat metric, also called dual bounded Lipschitz distance, generalizes the well-known Wasserstein distance $W_1$ to the case that the distributions are of unequal…
The goal of this short note is to discuss the relation between Kullback--Leibler divergence and total variation distance, starting with the celebrated Pinsker's inequality relating the two, before switching to a simple, yet (arguably) more…
Variational Inference approximates an unnormalized distribution via the minimization of Kullback-Leibler (KL) divergence. Although this divergence is efficient for computation and has been widely used in applications, it suffers from some…
In this paper we study bounds for the total variation distance between two second degree polynomials in normal random variables provided that they essentially depend on at least three variables.
Distance covariance and distance correlation are scalar coefficients that characterize independence of random vectors in arbitrary dimension. Properties, extensions, and applications of distance correlation have been discussed in the recent…
Computing the similarity between two probability distributions is a recurring theme across control. We introduce a unified family of distances between the probability distributions of two random variables that is based on the discrepancy…
The ability to compute the exact divergence between two high-dimensional distributions is useful in many applications but doing so naively is intractable. Computing the alpha-beta divergence -- a family of divergences that includes the…
In a variety of applications it is important to extract information from a probability measure $\mu$ on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and possibly conditioned) continuous time…
In this note we explore how standard statistical distances are equivalent for discrete log-concave distributions. Distances include total variation distance, Wasserstein distance, and $f$-divergences.
Pinsker's classical inequality asserts that the total variation $TV(\mu, \nu)$ between two probability measures is bounded by $\sqrt{ 2H(\mu|\nu)}$ where $H$ denotes the relative entropy (or Kullback-Leibler divergence). Considering the…
In this article we demonstrate the relationship between finitely exchangeable arrays and finitely exchangeable sequences. We then derive sharp bounds on the total variation distance between distributions of finitely and infinitely…
The framework of approximate differential privacy is considered, and augmented by leveraging the notion of ``the total variation of a (privacy-preserving) mechanism'' (denoted by $\eta$-TV). With this refinement, an exact composition result…
$f$-divergences, which quantify discrepancy between probability distributions, are ubiquitous in information theory, machine learning, and statistics. While there are numerous methods for estimating $f$-divergences from data, a limit…
An important tool to quantify the likeness of two probability measures are f-divergences, which have seen widespread application in statistics and information theory. An example is the total variation, which plays an exceptional role among…
Let $\Gamma_n$ be an $n\times n$ Haar-invariant orthogonal matrix. Let $ Z_n$ be the $p\times q$ upper-left submatrix of $\Gamma_n,$ where $p=p_n$ and $q=q_n$ are two positive integers. Let $G_n$ be a $p\times q$ matrix whose $pq$ entries…