We present CadLLM, a training-free method to accelerate the inference throughput of diffusion-based LLMs (dLLMs). We first investigate the dynamic nature of token unmasking confidence across blocks and steps. Based on this observation, we present a lightweight adaptive approach that controls the generation block size, step size, and threshold based on the average confidence of unmasked tokens. We further reduce softmax overhead by dynamically leveraging a subset of the vocabulary to regulate sampling breadth. CadLLM is a plug-and-play, model-agnostic method compatible with KV-cache-based dLLMs. Extensive experiments on four popular tasks demonstrate that CadLLM yields up to 1.1-2.28x throughput improvement over the state-of-the-art baseline with competitive accuracy.
@article{arxiv.2512.07173,
title = {Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration},
author = {Jucheng Shen and Gaurav Sarkar and Yeonju Ro and Sharath Nittur Sridhar and Zhangyang Wang and Aditya Akella and Souvik Kundu},
journal= {arXiv preprint arXiv:2512.07173},
year = {2026}
}
Comments
12 pages, 3 figures. Accepted to Findings of ACL 2026