Yi-Lightning 技术报告
摘要
本技术报告介绍了Yi-Lightning，我们最新的旗舰大语言模型（LLM）。它在Chatbot Arena中overall ranking第6， 在专门类别中表现突出（包括中文、数学、编码和 Hard Prompts），分别排名第2至第4。Yi-Lightning采用增强型混合专家（MoE）架构， featuring advanced expert segmentation and routing mechanisms coupled with optimized KV-caching techniques。我们的开发过程包括 comprehensive pre-training、 supervised fine-tuning（SFT）和 reinforcement learning from human feedback（RLHF），其中我们构思了 multi-stage training、 synthetic data construction and reward modeling的 deliberate strategies。此外，我们实现了RAISE（负责任AI安全引擎）， a four-component framework to address safety issues across pre-training, post-training, and serving phases。受我们可扩展的 super-computing infrastructure的支持， all these innovations substantial reduce training、 deployment and inference costs while maintaining high-performance standards。通过在 public academic benchmarks上的进一步评估，Yi-Lightning在与顶级LLM的竞争中表现出竞争力，但我们注意到， traditional, static benchmark results与 real-world, dynamic human preferences之间存在显著差异。这一观察促使我们对 conventional benchmarks在指导开发更智能和强大AI systems for practical applications方面的 utility进行了 critical reassessment。Yi-Lightning现已通过我们的开发者平台提供，访问地址为https://platform.lingyiwanwu.com。
引用
@article{arxiv.2412.01253,
title = {Yi-Lightning Technical Report},
author = {Alan Wake and Bei Chen and C. X. Lv and Chao Li and Chengen Huang and Chenglin Cai and Chujie Zheng and Daniel Cooper and Fan Zhou and Feng Hu and Ge Zhang and Guoyin Wang and Heng Ji and Howard Qiu and Jiangcheng Zhu and Jun Tian and Katherine Su and Lihuan Zhang and Liying Li and Ming Song and Mou Li and Peng Liu and Qicheng Hu and Shawn Wang and Shijun Zhou and Shiming Yang and Shiyong Li and Tianhang Zhu and Wen Xie and Wenhao Huang and Xiang He and Xiaobo Chen and Xiaohui Hu and Xiaoyi Ren and Xinyao Niu and Yanpeng Li and Yongke Zhao and Yongzhen Luo and Yuchi Xu and Yuxuan Sha and Zhaodong Yan and Zhiyuan Liu and Zirui Zhang and Zonghong Dai},
journal= {arXiv preprint arXiv:2412.01253},
year = {2025}
}