Advances in Pre-Training Distributed Word Representations
Computation and Language
2017-12-29 v1
Abstract
Many Natural Language Processing applications nowadays rely on pre-trained word representations estimated from large text corpora such as news collections, Wikipedia and Web Crawl. In this paper, we show how to train high-quality word vector representations by using a combination of known tricks that are however rarely used together. The main result of our work is the new set of publicly available pre-trained models that outperform the current state of the art by a large margin on a number of tasks.
Cite
@article{arxiv.1712.09405,
title = {Advances in Pre-Training Distributed Word Representations},
author = {Tomas Mikolov and Edouard Grave and Piotr Bojanowski and Christian Puhrsch and Armand Joulin},
journal= {arXiv preprint arXiv:1712.09405},
year = {2017}
}