Combining Pretrained High-Resource Embeddings and Subword Representations for Low-Resource Languages
Computation and Language
2020-04-22 v3 Machine Learning
Abstract
The contrast between the need for large amounts of data for current Natural Language Processing (NLP) techniques, and the lack thereof, is accentuated in the case of African languages, most of which are considered low-resource. To help circumvent this issue, we explore techniques exploiting the qualities of morphologically rich languages (MRLs), while leveraging pretrained word vectors in well-resourced languages. In our exploration, we show that a meta-embedding approach combining both pretrained and morphologically-informed word embeddings performs best in the downstream task of Xhosa-English translation.
Cite
@article{arxiv.2003.04419,
title = {Combining Pretrained High-Resource Embeddings and Subword Representations for Low-Resource Languages},
author = {Machel Reid and Edison Marrese-Taylor and Yutaka Matsuo},
journal= {arXiv preprint arXiv:2003.04419},
year = {2020}
}
Comments
Accepted to the "AfricaNLP - Unlocking Local Languages" workshop at ICLR 2020