MiniMax-M1：通过闪电注意力高效扩展测试时计算
摘要
我们提出了 MiniMax-M1，世界上第一个开源权重的大规模混合注意力推理模型。MiniMax-M1 由混合专家架构和闪电注意力机制驱动。该模型基于我们之前的 MiniMax-Text-01 模型开发，该模型总共包含 4560 亿个参数，每个 token 激活 459 亿个参数。M1 模型原生支持 100 万 token 的上下文长度，是 DeepSeek R1 上下文大小的 8 倍。此外，MiniMax-M1 中的闪电注意力机制使得测试时计算能够高效扩展。这些特性使 M1 特别适合需要处理长输入和大量思考的复杂任务。MiniMax-M1 使用大规模强化学习 (RL) 在多样化问题上进行训练，包括基于沙箱的真实世界软件工程环境。除了 M1 在 RL 训练中的固有效率优势外，我们提出了 CISPO，一种进一步提高 RL 效率的新型 RL 算法。CISPO 裁剪重要性采样权重而非 token 更新，优于其他竞争性 RL 变体。混合注意力和 CISPO 的结合使 MiniMax-M1 的完整 RL 训练在 512 个 H800 GPU 上仅需三周完成，租赁成本仅为 534,700 美元。我们发布了两个版本的 MiniMax-M1 模型，分别具有 40K 和 80K 的思考预算，其中 40K 模型代表 80K 训练的中间阶段。在标准基准上的实验表明，我们的模型与强大的开源模型（如原始 DeepSeek-R1 和 Qwen3-235B）相当或更优，在复杂软件工程、工具利用和长上下文任务方面具有特别优势。我们在 https://github.com/MiniMax-AI/MiniMax-M1 公开发布 MiniMax-M1。
引用
@article{arxiv.2506.13585,
title = {MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention},
author = {MiniMax and : and Aili Chen and Aonian Li and Bangwei Gong and Binyang Jiang and Bo Fei and Bo Yang and Boji Shan and Changqing Yu and Chao Wang and Cheng Zhu and Chengjun Xiao and Chengyu Du and Chi Zhang and Chu Qiao and Chunhao Zhang and Chunhui Du and Congchao Guo and Da Chen and Deming Ding and Dianjun Sun and Dong Li and Enwei Jiao and Haigang Zhou and Haimo Zhang and Han Ding and Haohai Sun and Haoyu Feng and Huaiguang Cai and Haichao Zhu and Jian Sun and Jiaqi Zhuang and Jiaren Cai and Jiayuan Song and Jin Zhu and Jingyang Li and Jinhao Tian and Jinli Liu and Junhao Xu and Junjie Yan and Junteng Liu and Junxian He and Kaiyi Feng and Ke Yang and Kecheng Xiao and Le Han and Leyang Wang and Lianfei Yu and Liheng Feng and Lin Li and Lin Zheng and Linge Du and Lingyu Yang and Lunbin Zeng and Minghui Yu and Mingliang Tao and Mingyuan Chi and Mozhi Zhang and Mujie Lin and Nan Hu and Nongyu Di and Peng Gao and Pengfei Li and Pengyu Zhao and Qibing Ren and Qidi Xu and Qile Li and Qin Wang and Rong Tian and Ruitao Leng and Shaoxiang Chen and Shaoyu Chen and Shengmin Shi and Shitong Weng and Shuchang Guan and Shuqi Yu and Sichen Li and Songquan Zhu and Tengfei Li and Tianchi Cai and Tianrun Liang and Weiyu Cheng and Weize Kong and Wenkai Li and Xiancai Chen and Xiangjun Song and Xiao Luo and Xiao Su and Xiaobo Li and Xiaodong Han and Xinzhu Hou and Xuan Lu and Xun Zou and Xuyang Shen and Yan Gong and Yan Ma and Yang Wang and Yiqi Shi and Yiran Zhong and Yonghong Duan and Yongxiang Fu and Yongyi Hu and Yu Gao and Yuanxiang Fan and Yufeng Yang and Yuhao Li and Yulin Hu and Yunan Huang and Yunji Li and Yunzhi Xu and Yuxin Mao and Yuxuan Shi and Yuze Wenren and Zehan Li and Zelin Li and Zhanxu Tian and Zhengmao Zhu and Zhenhua Fan and Zhenzhen Wu and Zhichao Xu and Zhihang Yu and Zhiheng Lyu and Zhuo Jiang and Zibo Gao and Zijia Wu and Zijian Song and Zijun Sun},
journal= {arXiv preprint arXiv:2506.13585},
year = {2025}
}
备注
A technical report from MiniMax. The authors are listed in alphabetical order. We open-source our MiniMax-M1 at https://github.com/MiniMax-AI/MiniMax-M1