Inference scaling methods for LLMs often rely on decomposing problems into steps (or groups of tokens), followed by sampling and selecting the best next steps. However, these steps and their sizes are often predetermined or manually designed based on domain knowledge. We propose dynamic decomposition, a method that adaptively and automatically partitions solution and reasoning traces into manageable steps during inference. By more effectively allocating compute -- particularly through subdividing challenging steps and prioritizing their sampling -- dynamic decomposition significantly improves inference efficiency. Experiments on benchmarks such as APPS, MATH, and LiveCodeBench demonstrate that dynamic decomposition outperforms static approaches, including token-level, sentence-level, and single-step decompositions, reducing the pass@10 error rate by 5.0%, 6.7%, and 10.5% respectively. These findings highlight the potential of dynamic decomposition to improve a wide range of inference scaling techniques.
@article{arxiv.2502.16706,
title = {DISC: Dynamic Decomposition Improves LLM Inference Scaling},
author = {Jonathan Light and Wei Cheng and Benjamin Riviere and Wu Yue and Masafumi Oyamada and Mengdi Wang and Yisong Yue and Santiago Paternain and Haifeng Chen},
journal= {arXiv preprint arXiv:2502.16706},
year = {2025}
}
Comments
10 pages, Accepted to NeurIPS 2025 (Conference on Neural Information Processing Systems)