Entropy of Hidden Markov Processes via Cycle Expansion
Information Theory
2009-11-13 v1 Other Condensed Matter
math.IT
Data Analysis, Statistics and Probability
Abstract
Hidden Markov Processes (HMP) is one of the basic tools of the modern probabilistic modeling. The characterization of their entropy remains however an open problem. Here the entropy of HMP is calculated via the cycle expansion of the zeta-function, a method adopted from the theory of dynamical systems. For a class of HMP this method produces exact results both for the entropy and the moment-generating function. The latter allows to estimate, via the Chernoff bound, the probabilities of large deviations for the HMP. More generally, the method offers a representation of the moment-generating function and of the entropy via convergent series.
Keywords
Cite
@article{arxiv.0810.4341,
title = {Entropy of Hidden Markov Processes via Cycle Expansion},
author = {Armen E. Allahverdyan},
journal= {arXiv preprint arXiv:0810.4341},
year = {2009}
}
Comments
24 pages, 5 figures, published in Journal of Statistical Physics