English

Emergent Manifold Separability during Reasoning in Large Language Models

Machine Learning 2026-05-11 v2

Abstract

Chain-of-Thought (CoT) prompting significantly improves reasoning in Large Language Models, yet the temporal dynamics of the underlying representation geometry remain poorly understood. We investigate these dynamics by applying Manifold Capacity Theory (MCT) to two compositional reasoning tasks: a controlled Boolean logic tree that supports deep mechanistic analysis, and a natural-language eligibility task in which the model has to extract attributes from prose, compare them to thresholds, and compose the local decisions through a fixed evaluation tree. MCT lets us quantify the linear separability of latent representations without the confounding factors of probe training. On both tasks, and across several open-weight models, reasoning manifests as a transient geometric pulse: concept manifolds are untangled into linearly separable subspaces immediately prior to computation and rapidly compressed thereafter. This behavior diverges from standard linear probe accuracy, which remains high long after computation, suggesting a fundamental distinction between information that is merely retrievable and information that is geometrically prepared for processing. We interpret this phenomenon as Dynamic Manifold Management, a mechanism where the model dynamically modulates representational capacity to optimize the bandwidth of the residual stream throughout the reasoning chain.

Keywords

Cite

@article{arxiv.2602.20338,
  title  = {Emergent Manifold Separability during Reasoning in Large Language Models},
  author = {Chanwoo Chun and Alexandre Polo and SueYeon Chung},
  journal= {arXiv preprint arXiv:2602.20338},
  year   = {2026}
}

Comments

Alexandre Polo and Chanwoo Chun contributed equally to this work

R2 v1 2026-07-01T10:48:48.456Z