English

SetupX: Can LLM Agents Learn from Past Failures in Functionality-Correct Code Repository Setup?

Software Engineering 2026-05-28 v2 Artificial Intelligence Computation and Language Machine Learning

Abstract

Functionality-correct repository setup aims to configure execution environments (e.g., dependencies, build scripts) to successfully execute a repository's documented features. It presents significant challenges due to diverse, repository-specific failures, including dependency incompatibilities, missing toolchains, incomplete installations, and verification-strategy mismatches. Existing LLM agents struggle to robustly resolve these issues, specifically failing to support (1) cross-repository experience transfer, (2) multi-step trial-and-repair under non-invertible state changes, and (3) robust verification of setup outcomes to distinguish setup-induced failures from repository bugs. To address this, we introduce SetupX, an experiential learning-based setup framework. First, we construct a Self-Evolving Experience Representation (XPU), a dual-modality knowledge unit encoding setup signals, textual guidance, executable actions to dynamically transfer verified environment fixes to unseen repositories. Second, we employ Experience-Augmented Speculative Execution backed by a LIFO Docker snapshot stack, enabling the agent to proactively trial fixes and safely roll back to known-good states. Third, we introduce a Prosecutor-Judge Verification Protocol that separates evidence collection from final judgment, enabling more reliable setup verification beyond superficial build-time metrics. Evaluation results on carefully-crafted benchmarks show SetupX achieves highest performance (e.g., 92% pass rate) and outperforms the strongest baseline by over 19%. Crucially, SetupX excels in complex multi-repository setup requiring coordinating multiple interconnected services across different containers. The code repository is available at https://github.com/OpenDataBox/SetupX.

Keywords

Cite

@article{arxiv.2605.26186,
  title  = {SetupX: Can LLM Agents Learn from Past Failures in Functionality-Correct Code Repository Setup?},
  author = {Zihang Zhou and Ziqian Ren and Yukai Wu and Yingjie Xiong and Wei Zhou and Chao Peng and Dong Zhang and Bingheng Yan and Xuanhe Zhou and Fan Wu},
  journal= {arXiv preprint arXiv:2605.26186},
  year   = {2026}
}

Comments

21 pages, 6 figures