Statistical estimation of the Kullback-Leibler divergence
Statistics Theory
2019-07-02 v1 Statistics Theory
Abstract
Wide conditions are provided to guarantee asymptotic unbiasedness and L^2-consistency of the introduced estimates of the Kullback-Leibler divergence for probability measures in R^d having densities w.r.t. the Lebesgue measure. These estimates are constructed by means of two independent collections of i.i.d. observations and involve the specified k-nearest neighbor statistics. In particular, the established results are valid for estimates of the Kullback-Leibler divergence between any two Gaussian measures in R^d with nondegenerate covariance matrices. As a byproduct we obtain new statements concerning the Kozachenko-Leonenko estimators of the Shannon differential entropy.
Cite
@article{arxiv.1907.00196,
title = {Statistical estimation of the Kullback-Leibler divergence},
author = {Alexander Bulinski and Denis Dimitrov},
journal= {arXiv preprint arXiv:1907.00196},
year = {2019}
}