理解代码生成中链式思维有效性：实证与信息论分析
摘要
大型语言模型 (LLM) 在代码生成方面取得了强大的性能，但 Chain-of-Thought (CoT) 提示如何帮助 remains unclear. 我们present a systematic empirical and information-theoretic study of CoT effectiveness in neural code generation, evaluating five paradigms (Zero-Shot, Zero-Shot CoT, Self-Planning, Structured CoT, Reasoning-CoT) across six Python benchmarks, a multilingual benchmark with 12 programming languages, and six models from 7B to 480B parameters, using conditional mutual information as a conceptual lens. Our results show that externally guided CoT consistently outperforms direct generation, with structured methods improving Pass@1 by 5--12\% on average while using substantially fewer tokens than reflective reasoning, and that CoT benefits depend on language type systems and model capacity. We further find that reasoning \emph{quality} is critical: high-quality structured CoT from strong generators yields significantly higher accuracy than lightweight alternatives with the same template, whereas naive Zero-Shot CoT can even degrade performance. These findings provide practical guidance for choosing CoT strategies based on model capacity, language characteristics, and task complexity.
引用
@article{arxiv.2512.09679,
title = {Understanding Chain-of-Thought Effectiveness in Code Generation: An Empirical and Information-Theoretic Analysis},
author = {Naizhu Jin and Zhong Li and Guang Yang and Tian Zhang and Qingkai Zeng},
journal= {arXiv preprint arXiv:2512.09679},
year = {2025}
}