Entropy Estimates from Insufficient Samplings
Data Analysis, Statistics and Probability
2011-11-09 v2 Computational Physics
Abstract
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1 points in each box) they have exponentially small biases. In the low sampling regime the errors increase but are still much smaller than for most other estimators. One advantage is that our main estimators are given analytically, with explicitly known analytical formulas for the biases.
Cite
@article{arxiv.physics/0307138,
title = {Entropy Estimates from Insufficient Samplings},
author = {P. Grassberger},
journal= {arXiv preprint arXiv:physics/0307138},
year = {2011}
}
Comments
5 pages, including 2 figures; revised version with corrected figures