FormalSpecCpp: A Dataset of C++ Formal Specifications created using LLMs
Abstract
FormalSpecCpp is a dataset designed to fill the gap in standardized benchmarks for verifying formal specifications in C++ programs. To the best of our knowledge, this is the first comprehensive collection of C++ programs with well-defined preconditions and postconditions. It provides a structured benchmark for evaluating specification inference tools and testing theaccuracy of generated specifications. Researchers and developers can use this dataset to benchmark specification inference tools,fine-tune Large Language Models (LLMs) for automated specification generation, and analyze the role of formal specifications in improving program verification and automated testing. By making this dataset publicly available, we aim to advance research in program verification, specification inference, and AI-assisted software development. The dataset and the code are available at https://github.com/MadhuNimmo/FormalSpecCpp.
Cite
@article{arxiv.2502.15217,
title = {FormalSpecCpp: A Dataset of C++ Formal Specifications created using LLMs},
author = {Madhurima Chakraborty and Peter Pirkelbauer and Qing Yi},
journal= {arXiv preprint arXiv:2502.15217},
year = {2025}
}
Comments
Accepted at the 2025 IEEE/ACM 22nd International Conference on Mining Software Repositories (MSR)