面向低资源菲律宾语言的零样态跨语言命名实体识别的 Meta 预训练
摘要
Named-entity recognition (NER) in low-resource languages is usually tackled by finetuning very large multilingual LMs, an option that is often infeasible in memory- or latency-constrained settings. We ask whether small decoder LMs can be pretrained so that they adapt quickly and transfer zero-shot to languages unseen during pretraining. To this end we replace part of the autoregressive objective with first-order model-agnostic meta-learning (MAML). Tagalog and Cebuano are typologically similar yet structurally different in their actor/non-actor voice systems, and hence serve as a challenging test-bed. Across four model sizes (11 M - 570 M) MAML lifts zero-shot micro-F1 by 2-6 pp under head-only tuning and 1-3 pp after full tuning, while cutting convergence time by up to 8%. Gains are largest for single-token person entities that co-occur with Tagalog case particles si/ni, highlighting the importance of surface anchors.
引用
@article{arxiv.2509.02160,
title = {Meta-Pretraining for Zero-Shot Cross-Lingual Named Entity Recognition in Low-Resource Philippine Languages},
author = {David Demitri Africa and Suchir Salhan and Yuval Weiss and Paula Buttery and Richard Diehl Martinez},
journal= {arXiv preprint arXiv:2509.02160},
year = {2025}
}
备注
Accepted (poster) to 5th Workshop on Multilingual Representation Learning at EMNLP 2025