English

Beyond Static Cutoffs: One-Shot Dynamic Thresholding for Diffusion Language Models

Machine Learning 2026-02-17 v3

Abstract

Masked diffusion language models (MDLMs) are becoming competitive with their autoregressive counterparts but typically decode with fixed steps and sequential unmasking. To accelerate decoding, recent work such as Fast-dLLM enables parallel decoding via a static global confidence threshold, yet we observe strong block- and step-wise confidence fluctuations and, within a dataset, near-identical confidence trajectories across inputs as measured by cosine similarity. Motivated by these observations, we introduce One-Shot Dynamic Thresholding (OSDT), which calibrates thresholds on a single sequence and applies them to subsequent inputs with negligible overhead. On GPQA, GSM8K, and HumanEval, OSDT attains superior accuracy-throughput trade-offs (+24% tokens/s on GSM8K at the best accuracy, +45% on GPQA with comparable accuracy, and +50% on HumanEval with a modest accuracy gap). Beyond these results, our findings suggest broader opportunities to leverage reusable task-level confidence signatures for more general-purpose algorithmic and systems innovations in diffusion decoding.

Keywords

Cite

@article{arxiv.2511.02077,
  title  = {Beyond Static Cutoffs: One-Shot Dynamic Thresholding for Diffusion Language Models},
  author = {Jucheng Shen and Yeonju Ro},
  journal= {arXiv preprint arXiv:2511.02077},
  year   = {2026}
}

Comments

7 pages, NeurIPS 2025 Efficient Reasoning Workshop

R2 v1 2026-07-01T07:20:17.309Z