NP-Engine：通过可验证的人工NP问题提升大语言模型的优化推理能力
摘要
大型语言模型（LLM）在数学、编码、逻辑和谜题等任务中展现出强大的推理能力，尤其是通过基于可验证奖励的强化学习（Reinforcement Learning with Verifiable Rewards, RLVR）。然而，其解决更复杂优化问题——尤其是NP难问题的能力仍未得到充分探索。为弥合这一差距，我们提出NP-ENGINE，即首个针对NP难问题的综合性训练与评估框架。NP-ENGINE涵盖10类跨5个领域的任务，每个任务配备（i）可控的实例生成器、（ii）基于规则的验证器以及（iii）提供近似最优解作为基准的启发式求解器。该生成器-验证器-启发式管道实现了可扩展且可验证的层次化难度RLVR训练。我们还引入NP-BENCH，一个源自NP-ENGINE-DATA的基准，旨在评估LLM解决NP难水平推理问题的能力，关注可行性及解的质量。此外，我们 presented QWEN2.5-7B-NP，一个通过零RLVR结合课程学习在Qwen2.5-7B-Instruct上训练得到的模型，在NP-BENCH上超越GPT-4o，实现相同模型规模下的SOTA性能。Beyond in-domain tasks, we demonstrate that RLVR training on NP-ENGINE-DATA enables strong out-of-domain (OOD) generalization to reasoning tasks (logic, puzzles, math, and knowledge), as well as non-reasoning tasks such as instruction following. We also observe a scaling trend: increasing task diversity improves OOD generalization. These findings suggest that task-rich RLVR training is a promising direction for advancing LLM's reasoning ability, revealing new insights into the scaling laws of RLVR.
引用
@article{arxiv.2510.16476,
title = {NP-Engine: Empowering Optimization Reasoning in Large Language Models with Verifiable Synthetic NP Problems},
author = {Xiaozhe Li and Xinyu Fang and Shengyuan Ding and Linyang Li and Haodong Duan and Qingwen Liu and Kai Chen},
journal= {arXiv preprint arXiv:2510.16476},
year = {2025}
}