Ensemble estimation of multivariate f-divergence
Information Theory
2015-03-16 v2 math.IT
Abstract
f-divergence estimation is an important problem in the fields of information theory, machine learning, and statistics. While several divergence estimators exist, relatively few of their convergence rates are known. We derive the MSE convergence rate for a density plug-in estimator of f-divergence. Then by applying the theory of optimally weighted ensemble estimation, we derive a divergence estimator with a convergence rate of O(1/T) that is simple to implement and performs well in high dimensions. We validate our theoretical results with experiments.
Cite
@article{arxiv.1404.6230,
title = {Ensemble estimation of multivariate f-divergence},
author = {Kevin R. Moon and Alfred O. Hero},
journal= {arXiv preprint arXiv:1404.6230},
year = {2015}
}
Comments
14 pages, 6 figures, a condensed version of this paper was accepted to ISIT 2014, Version 2: Moved the proofs of the theorems from the main body to appendices at the end