From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence
Abstract
Large language models (LLMs) have fundamentally transformed automated software development by enabling direct translation of natural language descriptions into functional code, driving commercial adoption through tools like Github Copilot (Microsoft), Cursor (Anysphere), Trae (ByteDance), and Claude Code (Anthropic). While the field has evolved dramatically from rule-based systems to Transformer-based architectures, achieving performance improvements from single-digit to over 95\% success rates on benchmarks like HumanEval. In this work, we provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs, systematically examining the complete model life cycle from data curation to post-training through advanced prompting paradigms, code pre-training, supervised fine-tuning, reinforcement learning, and autonomous coding agents. We analyze the code capability of the general LLMs (GPT-4, Claude, LLaMA) and code-specialized LLMs (StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder), critically examining the techniques, design decisions, and trade-offs. Further, we articulate the research-practice gap between academic research (e.g., benchmarks and tasks) and real-world deployment (e.g., software-related code tasks), including code correctness, security, contextual awareness of large codebases, and integration with development workflows, and map promising research directions to practical needs. Last, we conduct a series of experiments to provide a comprehensive analysis of code pre-training, supervised fine-tuning, and reinforcement learning, covering scaling law, framework selection, hyperparameter sensitivity, model architectures, and dataset comparisons.
Cite
@article{arxiv.2511.18538,
title = {From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence},
author = {Jian Yang and Xianglong Liu and Weifeng Lv and Ken Deng and Shawn Guo and Lin Jing and Yizhi Li and Shark Liu and Xianzhen Luo and Yuyu Luo and Changzai Pan and Ensheng Shi and Yingshui Tan and Renshuai Tao and Jiajun Wu and Xianjie Wu and Zhenhe Wu and Daoguang Zan and Chenchen Zhang and Wei Zhang and He Zhu and Terry Yue Zhuo and Kerui Cao and Xianfu Cheng and Jun Dong and Shengjie Fang and Zhiwei Fei and Xiangyuan Guan and Qipeng Guo and Zhiguang Han and Joseph James and Tianqi Luo and Renyuan Li and Yuhang Li and Yiming Liang and Congnan Liu and Jiaheng Liu and Qian Liu and Ruitong Liu and Tyler Loakman and Xiangxin Meng and Chuang Peng and Tianhao Peng and Jiajun Shi and Mingjie Tang and Boyang Wang and Haowen Wang and Yunli Wang and Fanglin Xu and Zihan Xu and Fei Yuan and Ge Zhang and Jiayi Zhang and Xinhao Zhang and Wangchunshu Zhou and Hualei Zhu and King Zhu and Bryan Dai and Aishan Liu and Zhoujun Li and Chenghua Lin and Tianyu Liu and Chao Peng and Kai Shen and Libo Qin and Shuangyong Song and Zizheng Zhan and Jiajun Zhang and Jie Zhang and Zhaoxiang Zhang and Bo Zheng},
journal= {arXiv preprint arXiv:2511.18538},
year = {2025}
}