Decoding in Geometry: Alleviating Embedding-Space Crowding for Complex Reasoning
Abstract
Sampling-based decoding underlies complex reasoning in large language models (LLMs), where decoding strategies critically shape model behavior. Temperature- and truncation-based methods reshape the next-token distribution through global probability reweighting or thresholding to balance the quality-diversity tradeoff. However, they operate solely on token probabilities, ignoring fine-grained relationships among tokens in the embedding space. We uncover a novel phenomenon, embedding-space crowding, where the next-token distribution concentrates its probability mass on geometrically close tokens in the embedding space. We quantify crowding at multiple granularities and find a statistical association with reasoning success in mathematical problem solving. Motivated by this finding, we propose CraEG, a plug-and-play sampling method that mitigates crowding through geometry-guided reweighting. CraEG is training-free, single-pass, and compatible with standard sampling strategies. Experiments on multiple models and benchmarks demonstrate improved generation performance, with gains in robustness and diversity metrics.
Cite
@article{arxiv.2601.22536,
title = {Decoding in Geometry: Alleviating Embedding-Space Crowding for Complex Reasoning},
author = {Yixin Yang and Qingxiu Dong and Zhifang Sui},
journal= {arXiv preprint arXiv:2601.22536},
year = {2026}
}