English

Towards Repository-Level Program Verification with Large Language Models

Software Engineering 2025-10-01 v1 Artificial Intelligence Programming Languages

Abstract

Recent advancements in large language models (LLMs) suggest great promises in code and proof generations. However, scaling automated formal verification to real-world projects requires resolving cross-module dependencies and global contexts, which are crucial challenges overlooked by existing LLM-based methods with a special focus on targeting isolated, function-level verification tasks. To systematically explore and address the significant challenges of verifying entire software repositories, we introduce RVBench, the first verification benchmark explicitly designed for repository-level evaluation, constructed from four diverse and complex open-source Verus projects. We further introduce RagVerus, an extensible framework that synergizes retrieval-augmented generation with context-aware prompting to automate proof synthesis for multi-module repositories. RagVerus triples proof pass rates on existing benchmarks under constrained model inference budgets, and achieves a 27% relative improvement on the more challenging RVBench benchmark, demonstrating a scalable and sample-efficient verification solution.

Keywords

Cite

@article{arxiv.2509.25197,
  title  = {Towards Repository-Level Program Verification with Large Language Models},
  author = {Si Cheng Zhong and Xujie Si},
  journal= {arXiv preprint arXiv:2509.25197},
  year   = {2025}
}

Comments

Accepted to LMPL 2025

R2 v1 2026-07-01T06:05:30.256Z