[Background/Context] AI assistants like GitHub Copilot are transforming software engineering; several studies have highlighted productivity improvements. However, their impact on code quality, particularly in terms of maintainability, requires further investigation. [Objective/Aim] This study aims to examine the influence of AI assistants on software maintainability, specifically assessing how these tools affect the ability of developers to evolve code. [Method] We will conduct a two-phased controlled experiment involving professional developers. In Phase 1, developers will add a new feature to a Java project, with or without the aid of an AI assistant. Phase 2, a randomized controlled trial, will involve a different set of developers evolving random Phase 1 projects - working without AI assistants. We will employ Bayesian analysis to evaluate differences in completion time, perceived productivity, code quality, and test coverage.
@article{arxiv.2408.10758,
title = {Does Co-Development with AI Assistants Lead to More Maintainable Code? A Registered Report},
author = {Markus Borg and Dave Hewett and Donald Graham and Noric Couderc and Emma Söderberg and Luke Church and Dave Farley},
journal= {arXiv preprint arXiv:2408.10758},
year = {2024}
}
Comments
Accepted as In-Principal Acceptance (IPA) for a Stage 1 registration of the Registered Report Track at the 40th IEEE International Conference on Software Maintenance and Evolution (ICSME)