English

Entropy Bounds for Discrete Random Variables via Maximal Coupling

Information Theory 2016-11-17 v5 math.IT Probability

Abstract

This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal coupling, and they apply to discrete random variables which are defined over finite or countably infinite alphabets. Loosened versions of these bounds are demonstrated to reproduce some previously reported results. The use of the new bounds is exemplified for the Poisson approximation, where bounds on the local and total variation distances follow from Stein's method.

Keywords

Cite

@article{arxiv.1209.5259,
  title  = {Entropy Bounds for Discrete Random Variables via Maximal Coupling},
  author = {Igal Sason},
  journal= {arXiv preprint arXiv:1209.5259},
  year   = {2016}
}

Comments

Final version. Accepted to the IEEE Trans. on Information Theory, July 2013

R2 v1 2026-06-21T22:10:00.154Z