A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
Abstract
The remarkable success of Large Language Models (LLMs) has illuminated a promising pathway toward achieving Artificial General Intelligence for both academic and industrial communities, owing to their unprecedented performance across various applications. As LLMs continue to gain prominence in both research and commercial domains, their security and safety implications have become a growing concern, not only for researchers and corporations but also for every nation. Currently, existing surveys on LLM safety primarily focus on specific stages of the LLM lifecycle, e.g., deployment phase or fine-tuning phase, lacking a comprehensive understanding of the entire "lifechain" of LLMs. To address this gap, this paper introduces, for the first time, the concept of "full-stack" safety to systematically consider safety issues throughout the entire process of LLM training, deployment, and eventual commercialization. Compared to the off-the-shelf LLM safety surveys, our work demonstrates several distinctive advantages: (I) Comprehensive Perspective. We define the complete LLM lifecycle as encompassing data preparation, pre-training, post-training, deployment and final commercialization. To our knowledge, this represents the first safety survey to encompass the entire lifecycle of LLMs. (II) Extensive Literature Support. Our research is grounded in an exhaustive review of over 800+ papers, ensuring comprehensive coverage and systematic organization of security issues within a more holistic understanding. (III) Unique Insights. Through systematic literature analysis, we have developed reliable roadmaps and perspectives for each chapter. Our work identifies promising research directions, including safety in data generation, alignment techniques, model editing, and LLM-based agent systems. These insights provide valuable guidance for researchers pursuing future work in this field.
Keywords
Cite
@article{arxiv.2504.15585,
title = {A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment},
author = {Kun Wang and Guibin Zhang and Zhenhong Zhou and Jiahao Wu and Miao Yu and Shiqian Zhao and Chenlong Yin and Jinhu Fu and Yibo Yan and Hanjun Luo and Liang Lin and Zhihao Xu and Haolang Lu and Xinye Cao and Xinyun Zhou and Weifei Jin and Fanci Meng and Shicheng Xu and Junyuan Mao and Yu Wang and Hao Wu and Minghe Wang and Fan Zhang and Junfeng Fang and Wenjie Qu and Yue Liu and Chengwei Liu and Yifan Zhang and Qiankun Li and Chongye Guo and Yalan Qin and Zhaoxin Fan and Kai Wang and Yi Ding and Donghai Hong and Jiaming Ji and Yingxin Lai and Zitong Yu and Xinfeng Li and Yifan Jiang and Yanhui Li and Xinyu Deng and Junlin Wu and Dongxia Wang and Yihao Huang and Yufei Guo and Jen-tse Huang and Qiufeng Wang and Xiaolong Jin and Wenxuan Wang and Dongrui Liu and Yanwei Yue and Wenke Huang and Guancheng Wan and Heng Chang and Tianlin Li and Yi Yu and Chenghao Li and Jiawei Li and Lei Bai and Jie Zhang and Qing Guo and Jingyi Wang and Tianlong Chen and Joey Tianyi Zhou and Xiaojun Jia and Weisong Sun and Cong Wu and Jing Chen and Xuming Hu and Yiming Li and Xiao Wang and Ningyu Zhang and Luu Anh Tuan and Guowen Xu and Jiaheng Zhang and Tianwei Zhang and Xingjun Ma and Jindong Gu and Liang Pang and Xiang Wang and Bo An and Jun Sun and Mohit Bansal and Shirui Pan and Lingjuan Lyu and Yuval Elovici and Bhavya Kailkhura and Yaodong Yang and Hongwei Li and Wenyuan Xu and Yizhou Sun and Wei Wang and Qing Li and Ke Tang and Yu-Gang Jiang and Felix Juefei-Xu and Hui Xiong and Xiaofeng Wang and Dacheng Tao and Philip S. Yu and Qingsong Wen and Yang Liu},
journal= {arXiv preprint arXiv:2504.15585},
year = {2025}
}