English

On the Plasticity and Stability for Post-Training Large Language Models

Machine Learning 2026-02-09 v1

Abstract

Training stability remains a critical bottleneck for Group Relative Policy Optimization (GRPO), often manifesting as a trade-off between reasoning plasticity and general capability retention. We identify a root cause as the geometric conflict between plasticity and stability gradients, which leads to destructive interference. Crucially, we argue that deterministic projection methods are suboptimal for GRPO as they overlook the intrinsic stochasticity of group-based gradient estimates. To address this, we propose Probabilistic Conflict Resolution (PCR), a Bayesian framework that models gradients as random variables. PCR dynamically arbitrates conflicts via an uncertainty-aware ``soft projection'' mechanism, optimizing the signal-to-noise ratio. Extensive experiments demonstrate that PCR significantly smooths the training trajectory and achieves superior performance in various reasoning tasks.

Keywords

Cite

@article{arxiv.2602.06453,
  title  = {On the Plasticity and Stability for Post-Training Large Language Models},
  author = {Wenwen Qiang and Ziyin Gu and Jiahuan Zhou and Jie Hu and Jingyao Wang and Changwen Zheng and Hui Xiong},
  journal= {arXiv preprint arXiv:2602.06453},
  year   = {2026}
}
R2 v1 2026-07-01T10:23:50.403Z