Lemmatization is still not a trivial task for morphologically rich languages. Previous studies showed that hybrid architectures usually work better for these languages and can yield great results. This paper presents a hybrid lemmatizer utilizing both a neural model, dictionaries and hand-crafted rules. We introduce a hybrid architecture along with empirical results on a widely used Hungarian dataset. The presented methods are published as three HuSpaCy models.
Cite
@article{arxiv.2306.07636,
title = {Hybrid lemmatization in HuSpaCy},
author = {Péter Berkecz and György Orosz and Zsolt Szántó and Gergő Szabó and Richárd Farkas},
journal= {arXiv preprint arXiv:2306.07636},
year = {2023}
}
Comments
published at the conference XIX. Magyar Sz\'am\'it\'og\'epes Nyelv\'eszeti Konferencia (XIX. Hungarian Computational Linguistics Conference)