Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. However, due to the bidirectional attention mechanism, DLMs incur substantial computational cost as context length increases. This work addresses this issue with a key discovery: unlike the sequential generation in autoregressive language models (ARLMs), the diffusion generation paradigm in DLMs allows \textit{efficient dynamic adjustment of the context} during generation. Building on this insight, we propose \textbf{D}ynamic \textbf{I}n-Context \textbf{P}lanner (DIP), a context-optimization method that dynamically selects and inserts in-context examples during generation, rather than providing all examples in the prompt upfront. Results show DIP maintains generation quality while achieving up to 12.9× inference speedup over standard inference and 1.17× over KV cache-enhanced inference.
@article{arxiv.2601.03199,
title = {DIP: Dynamic In-Context Planner For Diffusion Language Models},
author = {Yang Li and Han Meng and Chenan Wang and Haipeng Chen},
journal= {arXiv preprint arXiv:2601.03199},
year = {2026}
}