AfriMTE 与 AfriCOMET：增强 COMET 以涵盖资源匮乏的非洲语言
计算与语言
2024-04-25 v3
摘要
尽管近期在多语言机器翻译（MT）扩展到若干资源匮乏的非洲语言方面取得了进展，准确衡量这一进展仍具挑战，因为评估常基于 BLEU 等 n-gram 匹配指标，其通常与人类判断相关性较弱。诸如 COMET 等习得指标具有更高相关性；然而，缺乏带人类评分的非洲语言评估数据、诸如多维质量指标（MQM）等标注指南的复杂性，以及多语言编码器有限的语言覆盖，阻碍了它们在非洲语言上的适用性。本文中，我们通过为 13 种类型学多样的非洲语言创建带简化 MQM 指南用于错误检测的高质量人类评估数据和直接评估（DA）评分，应对这些挑战。此外，我们开发 AfriCOMET：通过利用来自资源充裕语言的 DA 数据和以非洲为中心的多语言编码器（AfroXLM-R），创建针对非洲语言的 COMET 评估指标，从而就与人类判断的 Spearman 秩相关（0.441）而言，成为非洲语言上最先进的 MT 评估指标。
引用
@article{arxiv.2311.09828,
title = {AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages},
author = {Jiayi Wang and David Ifeoluwa Adelani and Sweta Agrawal and Marek Masiak and Ricardo Rei and Eleftheria Briakou and Marine Carpuat and Xuanli He and Sofia Bourhim and Andiswa Bukula and Muhidin Mohamed and Temitayo Olatoye and Tosin Adewumi and Hamam Mokayed and Christine Mwase and Wangui Kimotho and Foutse Yuehgoh and Anuoluwapo Aremu and Jessica Ojo and Shamsuddeen Hassan Muhammad and Salomey Osei and Abdul-Hakeem Omotayo and Chiamaka Chukwuneke and Perez Ogayo and Oumaima Hourrane and Salma El Anigri and Lolwethu Ndolela and Thabiso Mangwana and Shafie Abdi Mohamed and Ayinde Hassan and Oluwabusayo Olufunke Awoyomi and Lama Alkhaled and Sana Al-Azzawi and Naome A. Etori and Millicent Ochieng and Clemencia Siro and Samuel Njoroge and Eric Muchiri and Wangari Kimotho and Lyse Naomi Wamba Momo and Daud Abolade and Simbiat Ajao and Iyanuoluwa Shode and Ricky Macharm and Ruqayya Nasir Iro and Saheed S. Abdullahi and Stephen E. Moore and Bernard Opoku and Zainab Akinjobi and Abeeb Afolabi and Nnaemeka Obiefuna and Onyekachi Raphael Ogbu and Sam Brian and Verrah Akinyi Otiende and Chinedu Emmanuel Mbonu and Sakayo Toadoum Sari and Yao Lu and Pontus Stenetorp},
journal= {arXiv preprint arXiv:2311.09828},
year = {2024}
}
备注
Accepted by NAACL 2024