English
Related papers

Related papers: Minimax Optimal Estimation of KL Divergence for Co…

200 papers

In this paper, we study the strong consistency of a bias reduced kernel density estimator and derive a strongly con- sistent Kullback-Leibler divergence (KLD) estimator. As application, we formulate a goodness-of-fit test and an…

Methodology · Statistics 2018-05-21 Papa Ngom , Freedath Djibril Moussa , Jean de Dieu Nkurunziza

Estimating Kullback Leibler (KL) divergence from samples of two distributions is essential in many machine learning problems. Variational methods using neural network discriminator have been proposed to achieve this task in a scalable…

Machine Learning · Computer Science 2021-10-01 Sandesh Ghimire , Aria Masoomi , Jennifer Dy

We study the problem of spectrum estimation from transmission data of a known phantom. The goal is to reconstruct an x-ray spectrum that can accurately model the x-ray transmission curves and reflects a realistic shape of the typical energy…

Medical Physics · Physics 2021-07-14 Wooseok Ha , Emil Y. Sidky , Rina Foygel Barber , Taly Gilat Schmidt , Xiaochuan Pan

Statistical divergences (SDs), which quantify the dissimilarity between probability distributions, are a basic constituent of statistical inference and machine learning. A modern method for estimating those divergences relies on…

Statistics Theory · Mathematics 2022-03-30 Sreejith Sreekumar , Ziv Goldfeld

Accelerated algorithms for maximum likelihood image reconstruction are essential for emerging applications such as 3D tomography, dynamic tomographic imaging, and other high dimensional inverse problems. In this paper, we introduce and…

Computation · Statistics 2012-01-31 Stéphane Chrétien , Alfred O. Hero

The Kullback-Leibler (KL) divergence is a fundamental equation of information theory that quantifies the proximity of two probability distributions. Although difficult to understand by examining the equation, an intuition and understanding…

Information Theory · Computer Science 2014-04-09 Jonathon Shlens

This paper presents how to perform minimax optimal classification, regression, and density estimation based on fixed-$k$ nearest neighbor (NN) searches. We consider a distributed learning scenario, in which a massive dataset is split into…

Statistics Theory · Mathematics 2024-09-11 J. Jon Ryu , Young-Han Kim

This paper proposes a new family of lower and upper bounds on the minimum mean squared error (MMSE). The key idea is to minimize/maximize the MMSE subject to the constraint that the joint distribution of the input-output statistics lies in…

Information Theory · Computer Science 2020-06-09 Michael Fauß , Alex Dysto , H. Vincent Poor

Several scalable sample-based methods to compute the Kullback Leibler (KL) divergence between two distributions have been proposed and applied in large-scale machine learning models. While they have been found to be unstable, the…

Machine Learning · Computer Science 2021-09-07 Sandesh Ghimire , Prashnna K Gyawali , Linwei Wang

We study concentration inequalities for the Kullback--Leibler (KL) divergence between the empirical distribution and the true distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes…

Information Theory · Computer Science 2019-10-22 Jay Mardia , Jiantao Jiao , Ervin Tánczos , Robert D. Nowak , Tsachy Weissman

Optimum designs for parameter estimation in generalized regression models are standardly based on the Fisher information matrix (cf. Atkinson et al (2014) for a recent exposition). The corresponding optimality criteria are related to the…

Statistics Theory · Mathematics 2015-07-28 Katarína Burclová , Andrej Pázman

The identification of unsubtracted foreground residuals in the cosmic microwave background maps on large scales is of crucial importance for the analysis of polarization signals. These residuals add a non-Gaussian contribution to the data.…

Cosmology and Nongalactic Astrophysics · Physics 2015-07-06 Assaf Ben-David , Hao Liu , Andrew D. Jackson

Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic…

Mathematical Physics · Physics 2025-07-16 Gennaro Auricchio , Giovanni Brigati , Paolo Giudici , Giuseppe Toscani

Nonnegative matrix factorization (NMF) is a standard linear dimensionality reduction technique for nonnegative data sets. In order to measure the discrepancy between the input data and the low-rank approximation, the Kullback-Leibler (KL)…

Optimization and Control · Mathematics 2021-05-12 Le Thi Khanh Hien , Nicolas Gillis

The goal of this short note is to discuss the relation between Kullback--Leibler divergence and total variation distance, starting with the celebrated Pinsker's inequality relating the two, before switching to a simple, yet (arguably) more…

Probability · Mathematics 2023-08-03 Clément L. Canonne

We study the fundamental problem of estimating an unknown discrete distribution $p$ over $d$ symbols, given $n$ i.i.d. samples from the distribution. We are interested in minimizing the KL divergence between the true distribution and the…

Machine Learning · Statistics 2025-05-30 Jiayuan Ye , Vitaly Feldman , Kunal Talwar

We interpret likelihood-based test functions from a geometric perspective where the Kullback-Leibler (KL) divergence is adopted to quantify the distance from a distribution to another. Such a test function can be seen as a sub-Gaussian…

Information Theory · Computer Science 2021-01-05 Yan Wang

$\alpha$-posteriors and their variational approximations distort standard posterior inference by downweighting the likelihood and introducing variational approximation errors. We show that such distortions, if tuned appropriately, reduce…

Machine Learning · Statistics 2021-04-20 Marco Avella Medina , José Luis Montiel Olea , Cynthia Rush , Amilcar Velez

We study the problem of $K$-armed bandits with reward distributions belonging to a one-parameter exponential distribution family. In the literature, several criteria have been proposed to evaluate the performance of such algorithms,…

Machine Learning · Computer Science 2025-05-28 Hao Qin , Kwang-Sung Jun , Chicheng Zhang

We analyze the $\ell_1$ and $\ell_\infty$ convergence rates of k nearest neighbor density estimation method. Our analysis includes two different cases depending on whether the support set is bounded or not. In the first case, the…

Machine Learning · Statistics 2020-10-02 Puning Zhao , Lifeng Lai