English

Do LLMs Encode Functional Importance of Reasoning Tokens?

Computation and Language 2026-04-22 v3 Artificial Intelligence Machine Learning

Abstract

Large language models solve complex tasks by generating long reasoning chains, achieving higher accuracy at the cost of increased computational cost and reduced ability to isolate functionally relevant reasoning. Prior work on compact reasoning shortens such chains through probabilistic sampling, heuristics, or supervision from frontier models, but offers limited insight into whether models internally encode token-level functional importance for answer generation. We address this gap diagnostically and propose greedy pruning, a likelihood-preserving deletion procedure that iteratively removes reasoning tokens whose removal minimally degrades model likelihood under a specified objective, yielding length-controlled reasoning chains. We evaluate pruned reasoning in a distillation framework and show that students trained on pruned chains outperform a frontier-model-supervised compression baseline at matched reasoning lengths. Finally, our analysis reveals systematic pruning patterns and shows that attention scores can predict greedy pruning ranks, further suggesting that models encode a nontrivial functional importance structure over reasoning tokens.

Keywords

Cite

@article{arxiv.2601.03066,
  title  = {Do LLMs Encode Functional Importance of Reasoning Tokens?},
  author = {Janvijay Singh and Dilek Hakkani-Tür},
  journal= {arXiv preprint arXiv:2601.03066},
  year   = {2026}
}

Comments

Updated after ACL Main 2026 acceptance; 25 pages, 8 figures, 4 tables;

R2 v1 2026-07-01T08:52:43.699Z