OpenThoughts：面向推理模型的数据配方
机器学习
2025-06-06 v2
摘要
推理模型在数学、代码和科学等众多基准上取得了快速进展。然而，关于最佳训练配方仍存许多未解之谜，因为领先模型往往依赖于含有极少公开信息的专有数据集。为此，OpenThoughts 项目旨在创建用于训练推理模型的开源数据集。经过初始探索，我们的 OpenThoughts2-1M 数据集导致了 OpenThinker2-32B，这是首个在公共推理数据上训练的模型，能够在诸多推理基准（如 AIME 和 LiveCodeBench）上匹配 DeepSeek-R1-Distill-32B。随后，我们通过系统性地检验数据生成管道的每个步骤进行改进，进行 1000 多次受控实验，这导致了 OpenThoughts3。将管道扩展至 120 万个示例，并使用 QwQ-32B 作为教师模型，我们的 OpenThoughts3-7B 模型取得了最先进的结果：在 AIME 2025 上达到 53%，在 LiveCodeBench 06/24-01/25 上达到 51%，在 GPQA Diamond 上达到 54% - 分别比 DeepSeek-R1-Distill-Qwen-7B 提升了 15.3、17.2 和 20.5 个百分点。我们的所有数据集和模型均已公开于 https://openthoughts.ai。
引用
@article{arxiv.2506.04178,
title = {OpenThoughts: Data Recipes for Reasoning Models},
author = {Etash Guha and Ryan Marten and Sedrick Keh and Negin Raoof and Georgios Smyrnis and Hritik Bansal and Marianna Nezhurina and Jean Mercat and Trung Vu and Zayne Sprague and Ashima Suvarna and Benjamin Feuer and Liangyu Chen and Zaid Khan and Eric Frankel and Sachin Grover and Caroline Choi and Niklas Muennighoff and Shiye Su and Wanjia Zhao and John Yang and Shreyas Pimpalgaonkar and Kartik Sharma and Charlie Cheng-Jie Ji and Yichuan Deng and Sarah Pratt and Vivek Ramanujan and Jon Saad-Falcon and Jeffrey Li and Achal Dave and Alon Albalak and Kushal Arora and Blake Wulfe and Chinmay Hegde and Greg Durrett and Sewoong Oh and Mohit Bansal and Saadia Gabriel and Aditya Grover and Kai-Wei Chang and Vaishaal Shankar and Aaron Gokaslan and Mike A. Merrill and Tatsunori Hashimoto and Yejin Choi and Jenia Jitsev and Reinhard Heckel and Maheswaran Sathiamoorthy and Alexandros G. Dimakis and Ludwig Schmidt},
journal= {arXiv preprint arXiv:2506.04178},
year = {2025}
}
备注
https://www.openthoughts.ai/blog/ot3. arXiv admin note: text overlap with arXiv:2505.23754 by other authors