English

ATLAS: Automated Toolkit for Large-Scale Verified Code Synthesis

Software Engineering 2026-02-03 v2

Abstract

Large language models have become proficient at generating functional code, but ensuring the output truly matches the programmer's intent remains difficult. Testing improves trust, yet for safety-critical applications, formal verification provides the only true guarantees through machine-checked proofs. However, verified code remains scarce compared to mainstream languages or mathematical theorem proving, limiting LLM capabilities in this domain. We present ATLAS, an automated pipeline that synthesizes verified programs to address this data bottleneck. Applied to the TACO dataset of Python solutions to LeetCode-style problems, ATLAS generates 2.7K verified Dafny programs, each with high-quality specifications and machine-checked proofs. Through task decomposition, we extract 19K training examples. Fine-tuning Qwen 2.5 7B Coder on this data improves performance from 32.4% to 56.9% on DafnyBench and from 15.8% to 65.8% on DafnySynthesis, demonstrating that synthetic data generation is a viable path to scaling LLM capabilities for formal verification.

Keywords

Cite

@article{arxiv.2512.10173,
  title  = {ATLAS: Automated Toolkit for Large-Scale Verified Code Synthesis},
  author = {Mantas Baksys and Stefan Zetzsche and Olivier Bouissou and Remi Delmas and Soonho Kong and Sean B. Holden},
  journal= {arXiv preprint arXiv:2512.10173},
  year   = {2026}
}
R2 v1 2026-07-01T08:19:44.443Z