Entropy Message Passing
Machine Learning
2016-11-18 v6 Information Theory
math.IT
Abstract
The paper proposes a new message passing algorithm for cycle-free factor graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as sum-product message passing over the entropy semiring, which has previously appeared in automata theory. The primary use of EMP is to compute the entropy of a model. However, EMP can also be used to compute expressions that appear in expectation maximization and in gradient descent algorithms.
Cite
@article{arxiv.0906.2895,
title = {Entropy Message Passing},
author = {Velimir M. Ilic and Miomir S. Stankovic and Branimir T. Todorovic},
journal= {arXiv preprint arXiv:0906.2895},
year = {2016}
}
Comments
5 pages, 1 figure, to appear in IEEE Transactions on Information Theory