English

A Decomposition Perspective to Long-context Reasoning for LLMs

Computation and Language 2026-04-10 v1 Artificial Intelligence Machine Learning

Abstract

Long-context reasoning is essential for complex real-world applications, yet remains a significant challenge for Large Language Models (LLMs). Despite the rapid evolution in long-context reasoning, current research often overlooks the internal complexity of the long-context reasoning task itself. In this paper, we move beyond this holistic view and decompose long-context reasoning into a set of fundamental atomic skills, and we then automatically synthesize a suite of pseudo datasets, each explicitly targeting a specific atomic skill. Our empirical analysis confirms that proficiency in these atomic skills is strongly correlated with general long-text reasoning performance. Building on this insight, we employ reinforcement learning on these pseudo datasets to sharpen the model's atomic skills, in the hope of boosting its general long-context reasoning ability. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our approach: it outperforms a strong baseline by an average margin of 7.7\% (improving from 46.3\% to 54.0\%) across Loogle, Loong, LongBench-v2, BrowscompLong, Ruler-qa2, and MRCR.

Keywords

Cite

@article{arxiv.2604.07981,
  title  = {A Decomposition Perspective to Long-context Reasoning for LLMs},
  author = {Yanling Xiao and Huaibing Xie and Guoliang Zhao and Shihan Dou and Shaolei Wang and Yiting Liu and Nantao Zheng and Cheng Zhang and Pluto Zhou and Zhisong Zhang and Lemao Liu},
  journal= {arXiv preprint arXiv:2604.07981},
  year   = {2026}
}
R2 v1 2026-07-01T12:00:48.392Z