MiMo-V2-Flash技术报告
计算与语言
2026-01-09 v2 人工智能
摘要
我们present MiMo-V2-Flash，这是一个总参数为309B、活跃参数为15B的Mixture-of-Experts（MoE）模型，旨在实现快速、强大的推理和智能体能力。MiMo-V2-Flash采用交错的滑动窗口注意力（SWA）与全局注意力的混合注意力架构，滑动窗口长度为128个token，混合比例为5:1。该模型在27万亿个token上进行预训练，采用Multi-Token Prediction（MTP），native上下文长度为32k，随后扩展至256k。为高效扩展后训练计算，MiMo-V2-Flash引入一种新颖的Multi-Teacher On-Policy Distillation（MOPD）范式。在此框架中，领域专用教师（例如通过大规模强化学习训练）提供稠密且token级奖励，使学生模型能够完美掌握教师专长。MiMo-V2-Flash虽然仅使用约1/2和1/3的总参数，却与DeepSeek-V3.2和Kimi-K2等顶级开源模型相当。在推理期间，通过将MTP用作投机解码的草稿模型，MiMo-V2-Flash实现最高可达3.6的接受长度和2.6倍的解码加速。我们开源了模型权重以及三层MTP权重，以促进开放研究和社区合作。
引用
@article{arxiv.2601.02780,
title = {MiMo-V2-Flash Technical Report},
author = {Core Team and Bangjun Xiao and Bingquan Xia and Bo Yang and Bofei Gao and Bowen Shen and Chen Zhang and Chenhong He and Chiheng Lou and Fuli Luo and Gang Wang and Gang Xie and Hailin Zhang and Hanglong Lv and Hanyu Li and Heyu Chen and Hongshen Xu and Houbin Zhang and Huaqiu Liu and Jiangshan Duo and Jianyu Wei and Jiebao Xiao and Jinhao Dong and Jun Shi and Junhao Hu and Kainan Bao and Kang Zhou and Lei Li and Liang Zhao and Linghao Zhang and Peidian Li and Qianli Chen and Shaohui Liu and Shihua Yu and Shijie Cao and Shimao Chen and Shouqiu Yu and Shuo Liu and Tianling Zhou and Weijiang Su and Weikun Wang and Wenhan Ma and Xiangwei Deng and Bohan Mao and Bowen Ye and Can Cai and Chenghua Wang and Chengxuan Zhu and Chong Ma and Chun Chen and Chunan Li and Dawei Zhu and Deshan Xiao and Dong Zhang and Duo Zhang and Fangyue Liu and Feiyu Yang and Fengyuan Shi and Guoan Wang and Hao Tian and Hao Wu and Heng Qu and Hongfei Yi and Hongxu An and Hongyi Guan and Xing Zhang and Yifan Song and Yihan Yan and Yihao Zhao and Yingchun Lai and Yizhao Gao and Yu Cheng and Yuanyuan Tian and Yudong Wang and Zhen Tang and Zhengju Tang and Zhengtao Wen and Zhichao Song and Zhixian Zheng and Zihan Jiang and Jian Wen and Jiarui Sun and Jiawei Li and Jinlong Xue and Jun Xia and Kai Fang and Menghang Zhu and Nuo Chen and Qian Tu and Qihao Zhang and Qiying Wang and Rang Li and Rui Ma and Shaolei Zhang and Shengfan Wang and Shicheng Li and Shuhao Gu and Shuhuai Ren and Sirui Deng and Tao Guo and Tianyang Lu and Weiji Zhuang and Weikang Zhang and Weimin Xiong and Wenshan Huang and Wenyu Yang and Xin Zhang and Xing Yong and Xu Wang and Xueyang Xie and Yilin Jiang and Yixin Yang and Yongzhe He and Yu Tu and Yuanliang Dong and Yuchen Liu and Yue Ma and Yue Yu and Yuxing Xiang and Zhaojun Huang and Zhenru Lin and Zhipeng Xu and Zhiyang Chen and Zhonghua Deng and Zihan Zhang and Zihao Yue},
journal= {arXiv preprint arXiv:2601.02780},
year = {2026}
}
备注
31 pages, technical report