面向东南亚的包容性多语言 LLM —— Sailor2
计算与语言
2025-02-19 v1 人工智能
机器学习
摘要
Sailor2 是一个面向东南亚 (SEA) 语言的前沿多语言语言模型家族，提供 1B、8B 和 20B 三个规模版本，以适应不同应用。基于 Qwen2.5，Sailor2 通过持续预训练 5000 亿 token（其中 4000 亿 SEA-specific 和 1000 亿 replay tokens）支持 13 种 SEA 语言，同时保持对中文和英文的熟练程度。Sailor2-20B 模型在 SEA 语言上以 50-50 的赢率超越 GPT-4o。我们还提供了关于如何高效开发多语言模型的综合性烹饪书，包括五个关键方面：数据 curation、pre-training、post-training、model customization 和 evaluation。我们希望 Sailor2 模型（Apache 2.0 许可证）能推动 SEA 地区的语言发展，Sailor2 烹饪书也希望激励研究人员为其他资源不足的语言构建更具包容性的 LLM。
引用
@article{arxiv.2502.12982,
title = {Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs},
author = {Longxu Dou and Qian Liu and Fan Zhou and Changyu Chen and Zili Wang and Ziqi Jin and Zichen Liu and Tongyao Zhu and Cunxiao Du and Penghui Yang and Haonan Wang and Jiaheng Liu and Yongchi Zhao and Xiachong Feng and Xin Mao and Man Tsung Yeung and Kunat Pipatanakul and Fajri Koto and Min Si Thu and Hynek Kydlíček and Zeyi Liu and Qunshu Lin and Sittipong Sripaisarnmongkol and Kridtaphad Sae-Khow and Nirattisai Thongchim and Taechawat Konkaew and Narong Borijindargoon and Anh Dao and Matichon Maneegard and Phakphum Artkaew and Zheng-Xin Yong and Quan Nguyen and Wannaphong Phatthiyaphaibun and Hoang H. Tran and Mike Zhang and Shiqi Chen and Tianyu Pang and Chao Du and Xinyi Wan and Wei Lu and Min Lin},
journal= {arXiv preprint arXiv:2502.12982},
year = {2025}
}
备注
49 pages, 16 figures. Technical Report of Sailor2: https://sea-sailor.github.io/blog/sailor2/