English
Related papers

Related papers: Debugging Flaky Tests using Spectrum-based Fault L…

200 papers

Fault localization is a practical research topic that helps developers identify code locations that might cause bugs in a program. Most existing fault localization techniques are designed for imperative programs (e.g., C and Java) and rely…

Software Engineering · Computer Science 2021-02-23 Guolong Zheng , ThanhVu Nguyen , Simón Gutiérrez Brida , Germán Regis , Marcelo F. Frias , Nazareno Aguirre , Hamid Bagheri

Traditional spectrum-based fault localization (SBFL) exploits differences in a program's coverage spectrum when run on passing and failing test cases. However, such runs can provide a wealth of additional information beyond mere coverage.…

Software Engineering · Computer Science 2025-03-07 Julian Aron Prenner , Romain Robbes

Testing-based fault localization has been a research focus in software engineering in the past decades. It localizes faulty program elements based on a set of passing and failing test executions. Since whether a fault could be triggered and…

Software Engineering · Computer Science 2025-04-04 Yiqian Wu , Yujie Liu , Yi Yin , Muhan Zeng , Zhentao Ye , Xin Zhang , Yingfei Xiong , Lu Zhang

Non-deterministically passing and failing test cases, so-called flaky tests, have recently become a focus area of software engineering research. While this research focus has been met with some enthusiastic endorsement from industry, prior…

Software Engineering · Computer Science 2022-04-11 Martin Gruber , Gordon Fraser

Flaky tests, tests that pass or fail nondeterministically without changes to code or environment, pose a serious threat to software reliability. While classical software engineering has developed a rich body of dynamic and static techniques…

Software Engineering · Computer Science 2025-12-29 Dongchan Kim , Hamidreza Khoramrokh , Lei Zhang , Andriy Miranskyy

Fault localization (FL) is a critical step in debugging, which typically relies on repeated executions to pinpoint faulty code regions. However, repeated executions can be impractical in the presence of non-deterministic failures or high…

Software Engineering · Computer Science 2026-05-13 Ahmadreza Saboor Yaraghi , Golnaz Gharachorlu , Sakina Fatima , Lionel C. Briand , Ruiyuan Wan , Ruifeng Gao

Flaky tests (tests with non-deterministic outcomes) pose a major challenge for software testing. They are known to cause significant issues such as reducing the effectiveness and efficiency of testing and delaying software releases. In…

Software Engineering · Computer Science 2022-12-05 Shawn Rasheed , Amjed Tahir , Jens Dietrich , Negar Hashemi , Lu Zhang

Fault localization is a fundamental aspect of debugging, aiming to identify code regions likely responsible for failures. Traditional techniques primarily correlate statement execution with failures, yet program behavior is influenced by…

Software Engineering · Computer Science 2025-02-27 Marius Smytzek , Martin Eberlein , Lars Grunske , Andreas Zeller

Flaky tests cause significant problems as they can interrupt automated build processes that rely on all tests succeeding and undermine the trustworthiness of tests. Numerous causes of test flakiness have been identified, and program…

Software Engineering · Computer Science 2022-08-03 Jens Dietrich , Shawn Rasheed , Amjed Tahir

Fault localization is one of the most time-consuming and error-prone parts of software debugging. There are several tools for helping developers in the fault localization process, however, they mostly target programs written in Java and…

Software Engineering · Computer Science 2021-08-30 Qusay Idrees Sarhan , Attila Szatmari , Rajmond Toth , Arpad Beszedes

Test flakiness, a non-deterministic behavior of builds irrelevant to code changes, is a major and continuing impediment to delivering reliable software. The very few techniques for the automated repair of test flakiness are specifically…

Software Engineering · Computer Science 2024-04-16 Yang Chen , Reyhaneh Jabbarvand

Software systems have been continuously evolved and delivered with high quality due to the widespread adoption of automated tests. A recurring issue hurting this scenario is the presence of flaky tests, a test case that may pass or fail…

Software Engineering · Computer Science 2021-03-24 B. H. P. Camara , M. A. G. Silva , A. T. Endo , S. R. Vergilio

In recent years, software engineers have explored ways to assist quantum software programmers. Our goal in this paper is to continue this exploration and see if quantum software programmers deal with some problems plaguing classical…

Software Engineering · Computer Science 2023-12-05 Lei Zhang , Mahsa Radnejad , Andriy Miranskyy

Flaky tests pass and fail non-deterministically when run on the same version of code. Although many techniques have been proposed to detect, debug, and repair flaky tests, reproducing their failures remains a major challenge due to their…

Software Engineering · Computer Science 2026-05-22 Suzzana Rafi , Mahbub-Ul-Hoque Sumon , Md Erfan , Maruf Morshed Khan , August Shi , Wing Lam

Context: Fault localization (FL) is the key activity while debugging a program. Any improvement to this activity leads to significant improvement in total software development cost. There is an internal linkage between the program spectrum…

Software Engineering · Computer Science 2024-03-11 Saksham Sahai Srivastava , Arpita Dutta , Rajib Mall

Flaky tests exhibit non-deterministic behavior during execution and they may pass or fail without any changes to the program under test. Detecting and classifying these flaky tests is crucial for maintaining the robustness of automated test…

Software Engineering · Computer Science 2025-02-06 Riddhi More , Jeremy S. Bradbury

The role of regression testing in software testing is crucial as it ensures that any new modifications do not disrupt the existing functionality and behaviour of the software system. The desired outcome is for regression tests to yield…

Software Engineering · Computer Science 2025-06-09 Xin Sun , Daniel Ståhl , Kristian Sandahl

Fault localization is an essential step in the debugging process. Spectrum-Based Fault Localization (SBFL) is a popular fault localization family of techniques, utilizing code-coverage to predict suspicious lines of code. In this paper, we…

Software Engineering · Computer Science 2023-03-17 André Silva , Matias Martinez , Benjamin Danglot , Davide Ginelli , Martin Monperrus

Fault localization is a critical step in software maintenance. Yet, many existing techniques, such as Spectrum-Based Fault Localization (SBFL), rely heavily on the availability of fault-triggering tests to be effective. In practice,…

Software Engineering · Computer Science 2025-10-28 Md Nakhla Rafi , Lorena Barreto Simedo Pacheco , An Ran Chen , Jinqiu Yang , Tse-Hsun , Chen

Flaky tests are problematic because they non-deterministically pass or fail for the same software version under test, causing confusion and wasting development effort. While machine learning models have been used to predict flakiness and…

Software Engineering · Computer Science 2024-10-08 Sakina Fatima , Hadi Hemmati , Lionel Briand