DenseHMM: Learning Hidden Markov Models by Learning Dense Representations
Abstract
We propose DenseHMM - a modification of Hidden Markov Models (HMMs) that allows to learn dense representations of both the hidden states and the observables. Compared to the standard HMM, transition probabilities are not atomic but composed of these representations via kernelization. Our approach enables constraint-free and gradient-based optimization. We propose two optimization schemes that make use of this: a modification of the Baum-Welch algorithm and a direct co-occurrence optimization. The latter one is highly scalable and comes empirically without loss of performance compared to standard HMMs. We show that the non-linearity of the kernelization is crucial for the expressiveness of the representations. The properties of the DenseHMM like learned co-occurrences and log-likelihoods are studied empirically on synthetic and biomedical datasets.
Cite
@article{arxiv.2012.09783,
title = {DenseHMM: Learning Hidden Markov Models by Learning Dense Representations},
author = {Joachim Sicking and Maximilian Pintz and Maram Akila and Tim Wirtz},
journal= {arXiv preprint arXiv:2012.09783},
year = {2020}
}
Comments
Accepted at LMRL workshop at NeurIPS 2020. Code is available on: https://github.com/fraunhofer-iais/dense-hmm