LongCat-Flash技术报告
计算与语言
2025-09-22 v2 人工智能
分布式、并行与集群计算
机器学习
摘要
我们介绍LongCat-Flash，一个拥有5600亿参数的混合专家（MoE）语言模型，旨在兼顾计算效率和高级智能体能力。源于对可扩展效率的需求，LongCat-Flash采用了两种新颖设计：（a）零计算专家，它能够根据上下文需求动态分配计算预算，每个词元激活18.6B至31.3B（平均27B）参数，优化资源使用。（b）捷径连接MoE，它扩大了计算-通信重叠窗口，与同等规模的模型相比，在推理效率和吞吐量上展现出显著增益。我们开发了一个全面的用于大模型的扩展框架，该框架结合了超参数迁移、模型增长初始化、多管齐下的稳定性套件和确定性计算，以实现稳定且可复现的训练。值得注意的是，借助可扩展架构设计与基础设施工作之间的协同效应，我们在30天内完成了超过20万亿词元的模型训练，同时实现了超过每秒100个词元（TPS）的推理速度，每百万输出词元的成本为0.70美元。为将LongCat-Flash培养为智能体智能，我们在优化的数据混合体上进行了大规模预训练，随后针对推理、代码和指令进行了目标明确的中期和后期训练，并通过合成数据和工具使用任务进一步增强了其能力。综合评估表明，作为一个非思维链基础模型，LongCat-Flash在其他领先模型中展现出极具竞争力的性能，在智能体任务方面具有卓越优势。LongCat-Flash的模型检查点已开源，以促进社区研究。LongCat Chat: https://longcat.ai Hugging Face: https://huggingface.co/meituan-longcat GitHub: https://github.com/meituan-longcat
引用
@article{arxiv.2509.01322,
title = {LongCat-Flash Technical Report},
author = {Meituan LongCat Team and Bayan and Bei Li and Bingye Lei and Bo Wang and Bolin Rong and Chao Wang and Chao Zhang and Chen Gao and Chen Zhang and Cheng Sun and Chengcheng Han and Chenguang Xi and Chi Zhang and Chong Peng and Chuan Qin and Chuyu Zhang and Cong Chen and Congkui Wang and Dan Ma and Daoru Pan and Defei Bu and Dengchang Zhao and Deyang Kong and Dishan Liu and Feiye Huo and Fengcun Li and Fubao Zhang and Gan Dong and Gang Liu and Gang Xu and Ge Li and Guoqiang Tan and Guoyuan Lin and Haihang Jing and Haomin Fu and Haonan Yan and Haoxing Wen and Haozhe Zhao and Hong Liu and Hongmei Shi and Hongyan Hao and Hongyin Tang and Huantian Lv and Hui Su and Jiacheng Li and Jiahao Liu and Jiahuan Li and Jiajun Yang and Jiaming Wang and Jian Yang and Jianchao Tan and Jiaqi Sun and Jiaqi Zhang and Jiawei Fu and Jiawei Yang and Jiaxi Hu and Jiayu Qin and Jingang Wang and Jiyuan He and Jun Kuang and Junhui Mei and Kai Liang and Ke He and Kefeng Zhang and Keheng Wang and Keqing He and Liang Gao and Liang Shi and Lianhui Ma and Lin Qiu and Lingbin Kong and Lingtong Si and Linkun Lyu and Linsen Guo and Liqi Yang and Lizhi Yan and Mai Xia and Man Gao and Manyuan Zhang and Meng Zhou and Mengxia Shen and Mingxiang Tuo and Mingyang Zhu and Peiguang Li and Peng Pei and Peng Zhao and Pengcheng Jia and Pingwei Sun and Qi Gu and Qianyun Li and Qingyuan Li and Qiong Huang and Qiyuan Duan and Ran Meng and Rongxiang Weng and Ruichen Shao and Rumei Li and Shizhe Wu and Shuai Liang and Shuo Wang and Suogui Dang and Tao Fang and Tao Li and Tefeng Chen and Tianhao Bai and Tianhao Zhou and Tingwen Xie and Wei He and Wei Huang and Wei Liu and Wei Shi and Wei Wang and Wei Wu and Weikang Zhao and Wen Zan and Wenjie Shi and Xi Nan and Xi Su and Xiang Li and Xiang Mei and Xiangyang Ji and Xiangyu Xi and Xiangzhou Huang and Xianpeng Li and Xiao Fu and Xiao Liu and Xiao Wei and Xiaodong Cai and Xiaolong Chen and Xiaoqing Liu and Xiaotong Li and Xiaowei Shi and Xiaoyu Li and Xili Wang and Xin Chen and Xing Hu and Xingyu Miao and Xinyan He and Xuemiao Zhang and Xueyuan Hao and Xuezhi Cao and Xunliang Cai and Xurui Yang and Yan Feng and Yang Bai and Yang Chen and Yang Yang and Yaqi Huo and Yerui Sun and Yifan Lu and Yifan Zhang and Yipeng Zang and Yitao Zhai and Yiyang Li and Yongjing Yin and Yongkang Lv and Yongwei Zhou and Yu Yang and Yuchen Xie and Yueqing Sun and Yuewen Zheng and Yuhuai Wei and Yulei Qian and Yunfan Liang and Yunfang Tai and Yunke Zhao and Zeyang Yu and Zhao Zhang and Zhaohua Yang and Zhenchao Zhang and Zhikang Xia and Zhiye Zou and Zhizhao Zeng and Zhongda Su and Zhuofan Chen and Zijian Zhang and Ziwen Wang and Zixu Jiang and Zizhe Zhao and Zongyu Wang and Zunhai Su},
journal= {arXiv preprint arXiv:2509.01322},
year = {2025}
}