English

Don't Commit Alone: Joint Token Commitment in Diffusion Large Language Models

Computation and Language 2026-07-05 v1

Abstract

Diffusion large language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from the shared context; when those positions are dependent, the resulting factorization error is captured by conditional total correlation, which confidence-based selection cannot observe from marginals alone. We propose CoCommit, a marker-gated coordination pass that briefly defers commitment: after the usual bundle selection, a learned marker announces the commit set and the backbone's last-nn layers are re-applied so marked positions coordinate -- approximating joint-mode decoding -- before greedy argmax writes tokens. The method reuses existing weights with one extra partial forward pass and no auxiliary model. On LLaDA2.1-mini with LoRA adapters and matched greedy inference, joint commitment improves accuracy on all six benchmarks we evaluate, with the largest gains on reasoning and exact-answer tasks.

Cite

@article{arxiv.2607.04469,
  title  = {Don't Commit Alone: Joint Token Commitment in Diffusion Large Language Models},
  author = {Lin Yao},
  journal= {arXiv preprint arXiv:2607.04469},
  year   = {2026}
}