Scaling reasoning capabilities beyond traditional domains such as math and coding is hindered by the lack of diverse and high-quality questions. To overcome this limitation, we introduce a scalable approach for generating diverse and challenging reasoning questions, accompanied by reference answers. We present NaturalReasoning, a comprehensive dataset comprising 2.8 million questions that span multiple domains, including STEM fields (e.g., Physics, Computer Science), Economics, Social Sciences, and more. We demonstrate the utility of the questions in NaturalReasoning through knowledge distillation experiments which show that NaturalReasoning can effectively elicit and transfer reasoning capabilities from a strong teacher model. Furthermore, we demonstrate that NaturalReasoning is also effective for unsupervised self-training using external reward models or self-rewarding. To foster future work, we publicly release NaturalReasoning at https://huggingface.co/datasets/facebook/natural_reasoning.
@article{arxiv.2502.13124,
title = {NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions},
author = {Weizhe Yuan and Jane Yu and Song Jiang and Karthik Padthe and Yang Li and Ilia Kulikov and Kyunghyun Cho and Dong Wang and Yuandong Tian and Jason E Weston and Xian Li},
journal= {arXiv preprint arXiv:2502.13124},
year = {2025}
}
Comments
Dataset at https://huggingface.co/datasets/facebook/natural_reasoning