DND: Boosting Large Language Models with Dynamic Nested Depth
Abstract
We introduce Dynamic Nested Depth (DND), a novel method that improves performance for off-the-shelf LLMs by selecting critical tokens to reprocess in a nested depth manner. Specifically, at the end of the given transformer layer, DND identifies more critical tokens with a router and feeds them back for an extra round of processing, effectively ``reviewing" difficult tokens while avoiding redundant computation for easier ones. The dynamic selection mechanism is tailored for precise control via two novel strategies: a router controlling loss to enhance token selection distinguishability, and a threshold control scheme to ensure selection stability. We demonstrate the effectiveness of DND by directly integrating it into pre-trained dense and MoE models during a post-training phase. On diverse benchmarks, this approach boosts the performances of the dense Qwen3-1.7B by 1.88% and the MoE Qwen3-30B-A3B by 0.87%, all with a minimal parameter and computing increase.
Cite
@article{arxiv.2510.11001,
title = {DND: Boosting Large Language Models with Dynamic Nested Depth},
author = {Tieyuan Chen and Xiaodong Chen and Haoxing Chen and Zhenzhong Lan and Weiyao Lin and Jianguo Li},
journal= {arXiv preprint arXiv:2510.11001},
year = {2026}
}
Comments
Accepted by ICLR 2026