Baichuan 2：开放大规模语言模型
计算与语言
2025-04-18 v4
摘要
大语言模型（LLMs）仅基于少量自然语言指令示例，便在各种自然语言任务上展现出卓越性能，减少了对大量特征工程的需求。然而，多数强大的 LLM 为闭源或除英语外其他语言能力有限。在本技术报告中，我们提出 Baichuan 2，一系列大规模多语言语言模型，包含 70 亿和 130 亿参数，从零开始在 2.6 万亿词元上训练。Baichuan 2 在 MMLU、CMMLU、GSM8K 和 HumanEval 等公开基准上匹配或超越其他同规模开源模型。此外，Baichuan 2 在医学与法律等垂直领域表现出色。我们将发布所有预训练模型检查点，以裨益研究界更好地理解 Baichuan 2 的训练动态。
引用
@article{arxiv.2309.10305,
title = {Baichuan 2: Open Large-scale Language Models},
author = {Aiyuan Yang and Bin Xiao and Bingning Wang and Borong Zhang and Ce Bian and Chao Yin and Chenxu Lv and Da Pan and Dian Wang and Dong Yan and Fan Yang and Fei Deng and Feng Wang and Feng Liu and Guangwei Ai and Guosheng Dong and Haizhou Zhao and Hang Xu and Haoze Sun and Hongda Zhang and Hui Liu and Jiaming Ji and Jian Xie and JunTao Dai and Kun Fang and Lei Su and Liang Song and Lifeng Liu and Liyun Ru and Luyao Ma and Mang Wang and Mickel Liu and MingAn Lin and Nuolan Nie and Peidong Guo and Ruiyang Sun and Tao Zhang and Tianpeng Li and Tianyu Li and Wei Cheng and Weipeng Chen and Xiangrong Zeng and Xiaochuan Wang and Xiaoxi Chen and Xin Men and Xin Yu and Xuehai Pan and Yanjun Shen and Yiding Wang and Yiyu Li and Youxin Jiang and Yuchen Gao and Yupeng Zhang and Zenan Zhou and Zhiying Wu},
journal= {arXiv preprint arXiv:2309.10305},
year = {2025}
}
备注
Baichuan 2 technical report. Github: https://github.com/baichuan-inc/Baichuan2