A reproduction of Apple's bi-directional LSTM models for language identification in short strings
Computation and Language
2021-02-15 v1 Machine Learning
Abstract
Language Identification is the task of identifying a document's language. For applications like automatic spell checker selection, language identification must use very short strings such as text message fragments. In this work, we reproduce a language identification architecture that Apple briefly sketched in a blog post. We confirm the bi-LSTM model's performance and find that it outperforms current open-source language identifiers. We further find that its language identification mistakes are due to confusion between related languages.
Keywords
Cite
@article{arxiv.2102.06282,
title = {A reproduction of Apple's bi-directional LSTM models for language identification in short strings},
author = {Mads Toftrup and Søren Asger Sørensen and Manuel R. Ciosici and Ira Assent},
journal= {arXiv preprint arXiv:2102.06282},
year = {2021}
}
Comments
Will be presented at EACL 2021 SRW