Nemotron-Cascade：用于通用推理模型的级联强化学习扩展
摘要
构建具备强化学习(RL)能力的通用推理模型面临着巨大的跨领域异构性问题，包括推理时间响应长度和验证延迟的巨大差异。这类变异性使RL基础设施复杂化，减慢训练速度，并使训练课程（例如响应长度扩展）和超参数选择变得困难。本文提出了级联域内强化学习(Cascade RL)，以开发能够在指令模式和深度思考模式下运行的 Nemotron-Cascade，无需相对于仅思考模式的模型性能差距。 departing from conventional approaches that blend heterogeneous prompts from different domains, Cascade RL orchestrates sequential, domain-wise RL, reducing engineering complexity and delivering state-of-the-art performance across a wide range of benchmarks. Notably, RLHF for alignment, when used as a pre-step, boosts the model's reasoning ability far beyond mere preference optimization, and subsequent domain-wise RLVR stages rarely degrade the benchmark performance attained in earlier domains and may even improve it (see an illustration in Figure 1)。Our 14B model, after RL, outperforms its SFT teacher, DeepSeek-R1-0528, on LiveCodeBench v5/v6/Pro and achieves silver-medal performance in the 2025 International Olympiad in Informatics (IOI)。We transparently share our training and data recipes。
引用
@article{arxiv.2512.13607,
title = {Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models},
author = {Boxin Wang and Chankyu Lee and Nayeon Lee and Sheng-Chieh Lin and Wenliang Dai and Yang Chen and Yangyi Chen and Zhuolin Yang and Zihan Liu and Mohammad Shoeybi and Bryan Catanzaro and Wei Ping},
journal= {arXiv preprint arXiv:2512.13607},
year = {2026}
}
备注
We publicly release the Nemotron-Cascade models and the full collection of training data at: https://huggingface.co/collections/nvidia/nemotron-cascade