LLM-Based Invariant Testing for Software Functional Bugs
Abstract
Manually writing unit tests to uncover functional bugs in software libraries is not only time-consuming but also requires a deep understanding of the intended semantics of the APIs. Heuristic-based test generation methods suffer from low usability because they cannot reason about program semantics or interpret source code and documentation as humans do. Traditional fuzzing techniques like OSS-Fuzz often rely on crashes to detect bugs, but functional bugs do not always cause crashes. To overcome these limitations, we present LISA, a novel LLM-based invariant testing framework for software functional bugs. LISA iteratively generates API sequences and program invariants guided by API n-gram feedback, achieving higher bug-detection rates and competitive code coverage compared with both fuzzing and prior LLM-based test generation approaches, and reporting each finding as a high-confidence bug candidate for developer confirmation.
Cite
@article{arxiv.2607.18711,
title = {LLM-Based Invariant Testing for Software Functional Bugs},
author = {Ruogu Yang and Yifeng He and Yundi Xu and Yuqing Wei and Hao Chen},
journal= {arXiv preprint arXiv:2607.18711},
year = {2026}
}
Comments
Accepted for publication at ISSRE 2026 Research Track