Assessing AI-Based Code Assistants in Method Generation Tasks

Vincenzo Corso; Leonardo Mariani; Daniela Micucci; Oliviero Riganelli

doi:10.1145/3639478.3643122

Assessing AI-Based Code Assistants in Method Generation Tasks

Software Engineering 2024-02-15 v1

Authors: Vincenzo Corso , Leonardo Mariani , Daniela Micucci , Oliviero Riganelli

View on arXiv ↗ PDF ↗ DOI ↗

Abstract

AI-based code assistants are increasingly popular as a means to enhance productivity and improve code quality. This study compares four AI-based code assistants, GitHub Copilot, Tabnine, ChatGPT, and Google Bard, in method generation tasks, assessing their ability to produce accurate, correct, and efficient code. Results show that code assistants are useful, with complementary capabilities, although they rarely generate ready-to-use correct code.

Keywords

software refactoring code generation

Cite

@article{arxiv.2402.09022,
  title  = {Assessing AI-Based Code Assistants in Method Generation Tasks},
  author = {Vincenzo Corso and Leonardo Mariani and Daniela Micucci and Oliviero Riganelli},
  journal= {arXiv preprint arXiv:2402.09022},
  year   = {2024}
}

Related papers

View all related →