Character-Word LSTM Language Models
Computation and Language
2017-04-11 v1
Abstract
We present a Character-Word Long Short-Term Memory Language Model which both reduces the perplexity with respect to a baseline word-level language model and reduces the number of parameters of the model. Character information can reveal structural (dis)similarities between words and can even be used when a word is out-of-vocabulary, thus improving the modeling of infrequent and unknown words. By concatenating word and character embeddings, we achieve up to 2.77% relative improvement on English compared to a baseline model with a similar amount of parameters and 4.57% on Dutch. Moreover, we also outperform baseline word-level models with a larger number of parameters.
Cite
@article{arxiv.1704.02813,
title = {Character-Word LSTM Language Models},
author = {Lyan Verwimp and Joris Pelemans and Hugo Van hamme and Patrick Wambacq},
journal= {arXiv preprint arXiv:1704.02813},
year = {2017}
}