Relative entropy via non-sequential recursive pair substitutions
Information Theory
2015-05-19 v2 Statistical Mechanics
math.IT
Abstract
The entropy of an ergodic source is the limit of properly rescaled 1-block entropies of sources obtained applying successive non-sequential recursive pairs substitutions (see P. Grassberger 2002 ArXiv:physics/0207023 and D. Benedetto, E. Caglioti and D. Gabrielli 2006 Jour. Stat. Mech. Theo. Exp. 09 doi:10.1088/1742.-5468/2006/09/P09011). In this paper we prove that the cross entropy and the Kullback-Leibler divergence can be obtained in a similar way.
Keywords
Cite
@article{arxiv.1007.3384,
title = {Relative entropy via non-sequential recursive pair substitutions},
author = {D. Benedetto and E. Caglioti and G. Cristadoro and M. Degli Esposti},
journal= {arXiv preprint arXiv:1007.3384},
year = {2015}
}
Comments
13 pages , 2 figures