Shannon's entropy revisited
Information Theory
2015-04-08 v1 math.IT
Abstract
I consider the effect of a finite sample size on the entropy of a sample of independent events. I propose formula for entropy which satisfies Shannon's axioms, and which reduces to Shannon's entropy when sample size is infinite. I discuss the physical meaning of the difference between two formulas, including some practical implications, such as maximum achievable channel utilization, and minimum achievable communication protocol overhead, for a given message size.
Keywords
Cite
@article{arxiv.1504.01407,
title = {Shannon's entropy revisited},
author = {Sergei Viznyuk},
journal= {arXiv preprint arXiv:1504.01407},
year = {2015}
}
Comments
5 pages, 1 figure