Related papers: Sharp Inequalities between Total Variation and Hel…
We give a proof that, under relatively mild conditions, fully-connected feed-forward deep random neural networks converge to a Gaussian mixture distribution as only the width of the last hidden layer goes to infinity. We conducted…
The goal of this short note is to discuss the relation between Kullback--Leibler divergence and total variation distance, starting with the celebrated Pinsker's inequality relating the two, before switching to a simple, yet (arguably) more…
We prove minimax bounds for estimating Gaussian location mixtures on $\mathbb{R}^d$ under the squared $L^2$ and the squared Hellinger loss functions. Under the squared $L^2$ loss, we prove that the minimax rate is upper and lower bounded by…
Given two high-dimensional Gaussians with the same mean, we prove a lower and an upper bound for their total variation distance, which are within a constant factor of one another.
Pinsker's classical inequality asserts that the total variation $TV(\mu, \nu)$ between two probability measures is bounded by $\sqrt{ 2H(\mu|\nu)}$ where $H$ denotes the relative entropy (or Kullback-Leibler divergence). Considering the…
In this article, we consider products of ergodic Markov chains and discuss their cutoffs in the total variation. Through a new inequality relating the total variation and the Hellinger distance, we may identify the total variation cutoffs…
Uncertainty propagation in non-linear dynamical systems has become a key problem in various fields including control theory and machine learning. In this work we focus on discrete-time non-linear stochastic dynamical systems. We present a…
Total variation distance (TV distance) is an important measure for the difference between two distributions. Recently, there has been progress in approximating the TV distance between product distributions: a deterministic algorithm for a…
Many scientific and engineering problems require to perform Bayesian inferences in function spaces, in which the unknowns are of infinite dimension. In such problems, choosing an appropriate prior distribution is an important task. In…
In this paper, we consider using total variation minimization to recover signals whose gradients have a sparse support, from a small number of measurements. We establish the proof for the performance guarantee of total variation (TV)…
In this paper, we establish a novel connection between total variation (TV) distance estimation and probabilistic inference. In particular, we present an efficient, structure-preserving reduction from relative approximation of TV distance…
This work considers the use of Total variation (TV) minimization in the recovery of a given gradient sparse vector from Gaussian linear measurements. It has been shown in recent studies that there exist a sharp phase transition behavior in…
We develop sharp bounds on the statistical distance between high-dimensional permutation mixtures and their i.i.d. counterparts. Our approach establishes a new geometric link between the spectrum of a complex channel overlap matrix and the…
This work introduces a new, explicit bound on the Hellinger distance between a continuous random variable and a Gaussian with matching mean and variance. As example applications, we derive a quantitative Hellinger central limit theorem and…
We quantify the parameter stability of a spherical Gaussian Mixture Model (sGMM) under small perturbations in distribution space. Namely, we derive the first explicit bound to show that for a mixture of spherical Gaussian $P$ (sGMM) in a…
Mixture distributions arise in many parametric and non-parametric settings -- for example, in Gaussian mixture models and in non-parametric estimation. It is often necessary to compute the entropy of a mixture, but, in most cases, this…
The Kullback-Leibler divergence, the Kullback-Leibler variation, and the Bernstein "norm" are used to quantify discrepancies among probability distributions in likelihood models such as nonparametric maximum likelihood and nonparametric…
$f$-divergences are a general class of divergences between probability measures which include as special cases many commonly used divergences in probability, mathematical statistics and information theory such as Kullback-Leibler…
We focus on Bayesian inverse problems with Gaussian likelihood, linear forward model, and priors that can be formulated as a Gaussian mixture. Such a mixture is expressed as an integral of Gaussian density functions weighted by a mixing…
We show that the square Hellinger distance between two Bayesian networks on the same directed graph, $G$, is subadditive with respect to the neighborhoods of $G$. Namely, if $P$ and $Q$ are the probability distributions defined by two…