English

LogSage: An LLM-Based Framework for CI/CD Failure Detection and Remediation with Industrial Validation

Software Engineering 2025-10-07 v2

Abstract

Continuous Integration and Deployment (CI/CD) pipelines are critical to modern software engineering, yet diagnosing and resolving their failures remains complex and labor-intensive. We present LogSage, the first end-to-end LLM-powered framework for root cause analysis (RCA) and automated remediation of CI/CD failures. LogSage employs a token-efficient log preprocessing pipeline to filter noise and extract critical errors, then performs structured diagnostic prompting for accurate RCA. For solution generation, it leverages retrieval-augmented generation (RAG) to reuse historical fixes and invokes automation fixes via LLM tool-calling. On a newly curated benchmark of 367 GitHub CI/CD failures, LogSage achieves over 98\% precision, near-perfect recall, and an F1 improvement of more than 38\% points in the RCA stage, compared with recent LLM-based baselines. In a year-long industrial deployment at ByteDance, it processed over 1.07M executions, with end-to-end precision exceeding 80\%. These results demonstrate that LogSage provides a scalable and practical solution for automating CI/CD failure management in real-world DevOps workflows.

Keywords

Cite

@article{arxiv.2506.03691,
  title  = {LogSage: An LLM-Based Framework for CI/CD Failure Detection and Remediation with Industrial Validation},
  author = {Weiyuan Xu and Juntao Luo and Tao Huang and Kaixin Sui and Jie Geng and Qijun Ma and Isami Akasaka and Xiaoxue Shi and Jing Tang and Peng Cai},
  journal= {arXiv preprint arXiv:2506.03691},
  year   = {2025}
}

Comments

12 pages, 8 figures