Coincidental Correctness in the Defects4J Benchmark
Abstract
Coincidental correctness (CC) arises when a defective program produces the correct output despite the fact that the defect within was exercised. Researchers have recognized the negative impact of coincidental correctness, and the authors have previously conducted a study demonstrating its prevalence in test suites. However, that study was limited to system tests and small subjects seeded with artificial defects. In this paper, we conduct a wider scope study of CC that addresses the following research questions in the context of the Defects4J benchmark: RQ1: Is CC prevalent in Defects4J? RQ2: Is CC affected by the testing levels in Defects4J? RQ3: Do CC tests induce peculiar infection paths in Defects4J? RQ4: Are the infections likely to be nullified within or outside the buggy method? ....
Keywords
Cite
@article{arxiv.1808.09233,
title = {Coincidental Correctness in the Defects4J Benchmark},
author = {Rawad Abou Assi and Chadi Trad and Marwan Maalouf and Wes Masri},
journal= {arXiv preprint arXiv:1808.09233},
year = {2019}
}
Comments
To appear in STVR