Bad Smells in Software Analytics Papers
Abstract
CONTEXT: There has been a rapid growth in the use of data analytics to underpin evidence-based software engineering. However the combination of complex techniques, diverse reporting standards and poorly understood underlying phenomena are causing some concern as to the reliability of studies. OBJECTIVE: Our goal is to provide guidance for producers and consumers of software analytics studies (computational experiments and correlation studies). METHOD: We propose using "bad smells", i.e., surface indications of deeper problems and popular in the agile software community and consider how they may be manifest in software analytics studies. RESULTS: We list 12 "bad smells" in software analytics papers (and show their impact by examples). CONCLUSIONS: We believe the metaphor of bad smell is a useful device. Therefore we encourage more debate on what contributes to the validty of software analytics studies (so we expect our list will mature over time).
Cite
@article{arxiv.1803.05518,
title = {Bad Smells in Software Analytics Papers},
author = {Tim Menzies and Martin Shepperd},
journal= {arXiv preprint arXiv:1803.05518},
year = {2019}
}
Comments
Accepted April 2019. To appear