MiroMind-M1：基于上下文感知多阶段策略优化的数学推理开源突破
摘要
大语言模型最近从 fluent text generation 向 diverse 领域的 advanced reasoning 迈进，催生了 reasoning language models。其中，数学推理作为 representative benchmark，因需 precise multi-step logic 和 abstract reasoning，可推广至其他任务。虽然 closed-source RLMs 如 GPT-o3 在 reasoning 能力方面表现突出，但其 proprietary nature 限制了 transparency 和 reproducibility。虽然许多 open-source 项目致力于弥合这一差距，但大多数项目在 not including critical resources（如 datasets 和 detailed training configurations）方面缺乏 sufficient openness，致使 reproducibility 受阻。为促进 RLM development 中的 transparency，我们引入 MiroMind-M1 系列，完全 open-source 的 RLMs 基于 Qwen-2.5 backbone，性能匹配或优于现有 open-source RLMs。具体而言，我们的模型在 two stages 训练：首先在 719K math-reasoning problems with verified CoT trajectories 上进行 SFT，随后在 62K 个 challenging and verifiable problems 上进行 RLVR。为 enhance RLVR process 的 robustness 和 efficiency，我们引入 Context-Aware Multi-Stage Policy Optimization，该算法将 length-progressive training 与 adaptive repetition penalty 集成，以鼓励 context-aware RL training。我们的模型在 AIME24、AIME25 和 MATH benchmarks 上实现 state-of-the-art 或 competitive performance，同时在 token efficiency 上表现突出。为 ensure reproducibility，我们发布 complete stack：模型（MiroMind-M1-SFT-7B、MiroMind-M1-RL-7B、MiroMind-M1-RL-32B）；数据集（MiroMind-M1-SFT-719K、MiroMind-M1-RL-62K）；以及所有 training and evaluation configurations。我们希望这些 resources 将 support further research and foster community advancement。
引用
@article{arxiv.2507.14683,
title = {MiroMind-M1: An Open-Source Advancement in Mathematical Reasoning via Context-Aware Multi-Stage Policy Optimization},
author = {Xingxuan Li and Yao Xiao and Dianwen Ng and Hai Ye and Yue Deng and Xiang Lin and Bin Wang and Zhanfeng Mo and Chong Zhang and Yueyi Zhang and Zonglin Yang and Ruilin Li and Lei Lei and Shihao Xu and Han Zhao and Weiling Chen and Feng Ji and Lidong Bing},
journal= {arXiv preprint arXiv:2507.14683},
year = {2025}
}
备注
Technical report