Connecting and Comparing Language Model Interpolation Techniques
Audio and Speech Processing
2019-08-27 v1 Computation and Language
Machine Learning
Sound
Abstract
In this work, we uncover a theoretical connection between two language model interpolation techniques, count merging and Bayesian interpolation. We compare these techniques as well as linear interpolation in three scenarios with abundant training data per component model. Consistent with prior work, we show that both count merging and Bayesian interpolation outperform linear interpolation. We include the first (to our knowledge) published comparison of count merging and Bayesian interpolation, showing that the two techniques perform similarly. Finally, we argue that other considerations will make Bayesian interpolation the preferred approach in most circumstances.
Cite
@article{arxiv.1908.09738,
title = {Connecting and Comparing Language Model Interpolation Techniques},
author = {Ernest Pusateri and Christophe Van Gysel and Rami Botros and Sameer Badaskar and Mirko Hannemann and Youssef Oualil and Ilya Oparin},
journal= {arXiv preprint arXiv:1908.09738},
year = {2019}
}