Considerations and Pitfalls in Controlled Experiments on Code Comprehension
Software Engineering
2021-03-17 v1
Abstract
Understanding program code is a complicated endeavor. As such, myriad different factors can influence the outcome. Investigations of program comprehension, and in particular those using controlled experiments, have to take these factors into account. In order to promote the development and use of sound experimental methodology, we discuss potential problems with regard to the experimental subjects, the code they work on, the tasks they are asked to perform, and the metrics for their performance.
Cite
@article{arxiv.2103.08769,
title = {Considerations and Pitfalls in Controlled Experiments on Code Comprehension},
author = {Dror G. Feitelson},
journal= {arXiv preprint arXiv:2103.08769},
year = {2021}
}
Comments
13 pages. Accepted to the 29th IEEE/ACM International Conference on Program Comprehension