English

Comparison of Current Approaches to Lemmatization: A Case Study in Estonian

Computation and Language 2024-04-24 v1

Abstract

This study evaluates three different lemmatization approaches to Estonian -- Generative character-level models, Pattern-based word-level classification models, and rule-based morphological analysis. According to our experiments, a significantly smaller Generative model consistently outperforms the Pattern-based classification model based on EstBERT. Additionally, we observe a relatively small overlap in errors made by all three models, indicating that an ensemble of different approaches could lead to improvements.

Keywords

Cite

@article{arxiv.2404.15003,
  title  = {Comparison of Current Approaches to Lemmatization: A Case Study in Estonian},
  author = {Aleksei Dorkin and Kairit Sirts},
  journal= {arXiv preprint arXiv:2404.15003},
  year   = {2024}
}

Comments

6 pages, 2 figures