A Probabilistic Upper Bound on Differential Entropy
Information Theory
2007-07-13 v1 math.IT
Abstract
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the unknown distribution is required, nor is the distribution required to have a density. Previous distribution-free bounds on the cumulative distribution function of a random variable given a sample of that variable are used to construct the bound. A simple, fast, and intuitive algorithm for computing the entropy bound from a sample is provided.
Cite
@article{arxiv.cs/0504091,
title = {A Probabilistic Upper Bound on Differential Entropy},
author = {Joseph DeStefano and Erik Learned-Miller},
journal= {arXiv preprint arXiv:cs/0504091},
year = {2007}
}