Software is now so large and complex that additional architecture is needed to guide theorem provers as they try to generate test suites. For example, the SNAP test suite generator (introduced in this paper) combines the Z3 theorem prover with the following tactic: sample around the average values seen in a few randomly selected valid tests. This tactic is remarkably effective. For 27 real-world programs with up to half a million variables, SNAP found test suites which were 10 to 750 smaller times than those found by the prior state-of-the-art. Also, SNAP ran orders of magnitude faster and (unlike prior work) generated 100% valid tests.
@article{arxiv.1905.05358,
title = {Building Very Small Test Suites (with Snap)},
author = {Jianfeng Chen and Xipeng Shen and Tim Menzies},
journal= {arXiv preprint arXiv:1905.05358},
year = {2020}
}
Comments
13 pages, 9 figures, submitted to IEEE Transactions on Software Engineering (Journal First)