$H(X)$ vs. $H(f(X))$
Information Theory
2017-04-25 v1 math.IT
Abstract
It is well known that the entropy of a finite random variable is always greater or equal to the entropy of a function of , with equality if and only if is one-to-one. In this paper, we give tights bounds on when the function is not one-to-one, and we illustrate a few scenarios where this matters. As an intermediate step towards our main result, we prove a lower bound on the entropy of a probability distribution, when only a bound on the ratio between the maximum and the minimum probability is known. Our lower bound improves previous results in the literature, and it could find applications outside the present scenario.
Keywords
Cite
@article{arxiv.1704.07059,
title = {$H(X)$ vs. $H(f(X))$},
author = {Ferdinando Cicalese and Luisa Gargano and Ugo Vaccaro},
journal= {arXiv preprint arXiv:1704.07059},
year = {2017}
}
Comments
To appear in ISIT 2017