English
Related papers

Related papers: Where Do AI Coding Agents Fail? An Empirical Study…

200 papers

Resolving team conflicts requires not only task-specific competence, but also social intelligence to find common ground and build consensus. As AI agents increasingly collaborate on complex work, they must develop coordination capabilities…

GitHub's Copilot for Pull Requests (PRs) is a promising service aiming to automate various developer tasks related to PRs, such as generating summaries of changes or providing complete walkthroughs with links to the relevant code. As this…

Software Engineering · Computer Science 2024-02-15 Tao Xiao , Hideaki Hata , Christoph Treude , Kenichi Matsumoto

In the first half of 2025, coding agents have emerged as a category of development tools that have very quickly transitioned to the practice. Unlike ''traditional'' code completion LLMs such as Copilot, agents like Cursor, Claude Code, or…

Software Engineering · Computer Science 2026-04-09 Romain Robbes , Théo Matricon , Thomas Degueule , Andre Hora , Stefano Zacchiroli

Software projects frequently use automation tools to perform repetitive activities in the distributed software development process. Recently, GitHub introduced GitHub Actions, a feature providing automated workflows for software projects.…

Software Engineering · Computer Science 2024-01-24 Mairieli Wessel , Joseph Vargovich , Marco A. Gerosa , Christoph Treude

In recent years, AI-based software engineering has progressed from pre-trained models to advanced agentic workflows, with Software Development Agents representing the next major leap. These agents, capable of reasoning, planning, and…

Software Engineering · Computer Science 2024-12-30 Zhi Chen , Lingxiao Jiang

In the pull-based development model, code contributions are submitted as pull requests (PRs) to undergo reviews and approval by other developers with the goal of being merged into the code base. A PR can be supported by a description, whose…

Software Engineering · Computer Science 2026-02-17 Shirin Pirouzkhah , Pavlína Wurzel Gonçalves , Alberto Bacchelli

The rapid adoption of large language models (LLMs) like ChatGPT has introduced new dynamics in software development, particularly within pull request workflows. While prior research has examined the quality of AI-generated code, less is…

Software Engineering · Computer Science 2026-04-07 Daniel Ogenrwot , John Businge

While prior work has examined the generation capabilities of Agentic AI systems, little is known about how reviewers respond to AI-authored code in practice. In this paper, we present a large-scale empirical study of code review dynamics in…

Software Engineering · Computer Science 2026-01-28 Md. Asif Haider , Thomas Zimmermann

Large language model (LLM) based coding agents increasingly act as autonomous contributors that generate and merge pull requests, yet their real-world effects on software projects are unclear-especially compared with widely adopted…

Software Engineering · Computer Science 2026-01-28 Shyam Agarwal , Hao He , Bogdan Vasilescu

Code review has evolved for decades, from informal peer checking to today's pull request (PR) workflows, yet it remains a largely manual, uneven, and cognitively demanding process. The rise of Artificial Intelligence (AI) coding assistants…

Software Engineering · Computer Science 2026-05-19 Hüseyin Özgür Kamalı , Erdem Tuna , Vahid Haratian , Eray Tüzün

Coding agents represent a new paradigm in automated software engineering, combining the reasoning capabilities of Large Language Models (LLMs) with tool-augmented interaction loops. However, coding agents still have severe limitations.…

Software Engineering · Computer Science 2026-04-06 Tural Mehtiyev , Wesley Assunção

The rise of large language models for code has reshaped software development. Autonomous coding agents, able to create branches, open pull requests, and perform code reviews, now actively contribute to real-world projects. Their growing…

Software Engineering · Computer Science 2026-04-02 Razvan Mihai Popescu , David Gros , Andrei Botocan , Rahul Pandita , Prem Devanbu , Maliheh Izadi

In pull-based development systems, code reviews and pull request comments play important roles in improving code quality. In such systems, reviewers attempt to carefully check a piece of code by different unit tests. Unfortunately,…

Software Engineering · Computer Science 2022-05-20 F. Khoshnoud , A. Rezaei Nasab , Z. Toudeji , A. Sami

As software engineering moves toward SE3.0, AI agents are increasingly used to carry out development tasks and contribute changes to software projects. It is therefore important to understand the extent of these contributions and how human…

Software Engineering · Computer Science 2026-01-29 Kazuma Yamasaki , Joseph Ayobami Joshua , Tasha Settewong , Mahmoud Alfadel , Kazumasa Shimari , Kenichi Matsumoto

AI coding agents increasingly act directly within software environments, yet existing analyses of their failures rely on benchmark trajectories that miss how developers actually experience misalignment. We present an observational study of…

Software Engineering · Computer Science 2026-05-29 Ningzhi Tang , Chaoran Chen , Gelei Xu , Yiyu Shi , Yu Huang , Collin McMillan , Tao Dong , Toby Jia-Jun Li

The software engineering community recently has witnessed widespread deployment of AI programming assistants, such as GitHub Copilot. However, in practice, developers do not accept AI programming assistants' initial suggestions at a high…

Software Engineering · Computer Science 2023-09-19 Jenny T. Liang , Chenyang Yang , Brad A. Myers

Pull-based development has enabled numerous volunteers to contribute to open-source projects with fewer barriers. Nevertheless, a considerable amount of pull requests (PRs) with valid contributions are abandoned by their contributors,…

Software Engineering · Computer Science 2024-08-23 SayedHassan Khatoonabadi , Diego Elias Costa , Rabe Abdalkareem , Emad Shihab

Code review is central to software engineering education but hard to scale in capstone projects due to tight deadlines, uneven peer feedback, and limited prior experience. We investigate an LLM-as-reviewer integrated directly into GitHub…

Software Engineering · Computer Science 2026-04-28 Eduardo Oliveira , Michael Fu , Patanamon Thongtanunam , Sonsoles López-Pernas , Mohammed Saqr

AI-based code review tools automatically review and comment on pull requests to improve code quality. Despite their growing presence, little is known about their actual impact. We present a large-scale empirical study of 16 popular AI-based…

Software Engineering · Computer Science 2026-04-28 Kexin Sun , Hongyu Kuang , Sebastian Baltes , Xin Zhou , He Zhang , Xiaoxing Ma , Guoping Rong , Dong Shao , Christoph Treude

As a maintainer of an open source software project, you are usually happy about contributions in the form of pull requests that bring the project a step forward. Past studies have shown that when reviewing a pull request, not only its…

Software Engineering · Computer Science 2021-03-08 Marvin Wyrich , Raoul Ghit , Tobias Haller , Christian Müller