BALSAM：面向阿拉伯语大语言模型的基准测试平台
计算与语言
2025-07-31 v1 人工智能
摘要
大语言模型（LLM）在英语领域取得了令人瞩目的进展，但这一进展并未在所有语言中得到同等体现。特别是，由于数据稀缺、阿拉伯语及其方言的语言多样性、形态复杂性等原因，LLM在阿拉伯语上的表现相对滞后。阿拉伯语基准测试的质量进一步阻碍了进展，这些基准测试通常依赖于静态的公开可用数据，缺乏全面的任务覆盖，或者没有提供带有盲测集的专用平台。这使得衡量实际进展和减轻数据污染变得困难。在此，我们旨在弥合这些差距。具体来说，我们引入了BALSAM，一个全面的、社区驱动的基准测试，旨在推进阿拉伯语LLM的开发和评估。它包含来自14个大类的78个NLP任务，共52K个样本，分为37K个测试集和15K个开发集，并提供了一个集中、透明的盲测平台。我们期望BALSAM成为一个统一的平台，能够设定标准并促进合作研究，以提升阿拉伯语LLM的能力。
引用
@article{arxiv.2507.22603,
title = {BALSAM: A Platform for Benchmarking Arabic Large Language Models},
author = {Rawan Al-Matham and Kareem Darwish and Raghad Al-Rasheed and Waad Alshammari and Muneera Alhoshan and Amal Almazrua and Asma Al Wazrah and Mais Alheraki and Firoj Alam and Preslav Nakov and Norah Alzahrani and Eman alBilali and Nizar Habash and Abdelrahman El-Sheikh and Muhammad Elmallah and Haonan Li and Hamdy Mubarak and Mohamed Anwar and Zaid Alyafeai and Ahmed Abdelali and Nora Altwairesh and Maram Hasanain and Abdulmohsen Al Thubaity and Shady Shehata and Bashar Alhafni and Injy Hamed and Go Inoue and Khalid Elmadani and Ossama Obeid and Fatima Haouari and Tamer Elsayed and Emad Alghamdi and Khalid Almubarak and Saied Alshahrani and Ola Aljarrah and Safa Alajlan and Areej Alshaqarawi and Maryam Alshihri and Sultana Alghurabi and Atikah Alzeghayer and Afrah Altamimi and Abdullah Alfaifi and Abdulrahman AlOsaimy},
journal= {arXiv preprint arXiv:2507.22603},
year = {2025}
}