English

Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models

Machine Learning 2026-05-28 v3 Artificial Intelligence

Abstract

While Diffusion Large Language Models (dLLMs) offer structural advantages for global planning, efficiently verifying that they arrive at correct answers via valid reasoning traces remains a critical challenge. In this work, we propose a geometric perspective: Reasoning on the Manifold. We hypothesize that valid generation trajectories reside as stable attractors on the high-density manifold of the learned distribution, whereas invalid paths exhibit off-manifold drift. To operationalize this, we introduce Bidirectional Manifold Consistency (BMC), a training-free, unsupervised metric that quantifies the stability of the generated sequence through a forward-masking and backward-reconstruction cycle. Empirically, we demonstrate BMC's versatility across the full reasoning lifecycle: (1) in Diagnosis, it serves as a robust discriminator of solution validity without ground truth answer; (2) in Inference, it enables rejection resampling to effectively concentrate computational resources on complex reasoning tasks; and (3) in Alignment, it functions as a dense geometric reward that transforms sparse outcome supervision into fine-grained guidance, empowering models to self-evolve beyond standard baselines. Our results establish intrinsic geometric stability as a robust indicator of correctness for dLLMs.

Keywords

Cite

@article{arxiv.2604.16565,
  title  = {Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models},
  author = {Jiaoyang Ruan and Xin Gao and Yinda Chen and Hengyu Zeng and Liang Du and Guanghao Li and Jie Fu and Jian Pu},
  journal= {arXiv preprint arXiv:2604.16565},
  year   = {2026}
}

Comments

31 pages, 7 figures. Accepted to the 43rd International Conference on Machine Learning (ICML 2026). Camera-ready version

R2 v1 2026-07-01T12:15:14.194Z