Reduction of Maximum Entropy Models to Hidden Markov Models
Artificial Intelligence
2013-01-07 v1 Computation and Language
Abstract
We show that maximum entropy (maxent) models can be modeled with certain kinds of HMMs, allowing us to construct maxent models with hidden variables, hidden state sequences, or other characteristics. The models can be trained using the forward-backward algorithm. While the results are primarily of theoretical interest, unifying apparently unrelated concepts, we also give experimental results for a maxent model with a hidden variable on a word disambiguation task; the model outperforms standard techniques.
Keywords
Cite
@article{arxiv.1301.0570,
title = {Reduction of Maximum Entropy Models to Hidden Markov Models},
author = {Joshua Goodman},
journal= {arXiv preprint arXiv:1301.0570},
year = {2013}
}
Comments
Appears in Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI2002)