English

APR: Penalizing Structural Redundancy in Large Reasoning Models via Anchor-based Process Rewards

Computation and Language 2026-02-10 v2

Abstract

Test-Time Scaling (TTS) has significantly enhanced the capabilities of Large Reasoning Models (LRMs) but introduces a critical side-effect known as Overthinking. We conduct a preliminary study to rethink this phenomenon from a fine-grained perspective. We observe that LRMs frequently conduct repetitive self-verification without revision even after obtaining the final answer during the reasoning process. We formally define this specific position where the answer first stabilizes as the Reasoning Anchor. By analyzing pre- and post-anchor reasoning behaviors, we uncover the structural redundancy fixed in LRMs: the meaningless repetitive verification after deriving the first complete answer, which we term the Answer-Stable Tail (AST). Motivated by this observation, we propose Anchor-based Process Reward (APR), a structure-aware reward shaping method that localizes the reasoning anchor and penalizes exclusively the post-anchor AST. Leveraging the policy optimization algorithm suitable for length penalties, our APR models achieved the performance-efficiency Pareto frontier at 1.5B and 7B scales averaged across five mathematical reasoning datasets while requiring substantially fewer computational resources for RL training.

Keywords

Cite

@article{arxiv.2602.00760,
  title  = {APR: Penalizing Structural Redundancy in Large Reasoning Models via Anchor-based Process Rewards},
  author = {Kaiyan Chang and Chenwei Zhu and Yingfeng Luo and Yifu Huo and Chenglong Wang and Xiaoqian Liu and Qiaozhi He and Tong Xiao and Zhengtao Yu and Jingbo Zhu},
  journal= {arXiv preprint arXiv:2602.00760},
  year   = {2026}
}

Comments

Under Review

R2 v1 2026-07-01T09:29:30.010Z