English
Related papers

Related papers: Where Do AI Coding Agents Fail? An Empirical Study…

200 papers

Context: Pull-based development model is widely used in open source, leading the trends in distributed software development. One aspect which has garnered significant attention is studies on pull request decision - identifying factors for…

Software Engineering · Computer Science 2021-05-31 Xunhui Zhang , Yue Yu , Georgios Gousios , Ayushi Rastogi

AI Agents have rapidly gained prominence in both research and industry as systems that extend large language models with planning, tool use, memory, and goal-directed action. Despite this progress, the development and maintenance of Agent…

Software Engineering · Computer Science 2026-01-27 Ali Asgari , Annibale Panichella , Pouria Derakhshanfar , Mitchell Olsthoorn

General-purpose agents perform tasks in unfamiliar environments without domain-specific manual customization. Yet no study has systematically measured how agent architecture shapes performance across heterogeneous protocols and diverse…

Failures in large-scale cloud systems incur substantial financial losses, making automated Root Cause Analysis (RCA) essential for operational stability. Recent efforts leverage Large Language Model (LLM) agents to automate this task, yet…

Artificial Intelligence · Computer Science 2026-03-05 Taeyoon Kim , Woohyeok Park , Hoyeong Yun , Kyungyong Lee

Background: Pull-based development has shaped the practice of Modern Code Review (MCR), in which reviewers can contribute code improvements, such as refactorings, through comments and commits in Pull Requests (PRs). Past MCR studies…

Software Engineering · Computer Science 2021-08-26 Flávia Coelho , Nikolaos Tsantalis , Tiago Massoni , Everton L. G. Alves

Large language models (LLMs) show promise for automating software development by translating requirements into code. However, even advanced prompting workflows like progressive prompting often leave some requirements unmet. Although methods…

Software Engineering · Computer Science 2026-02-04 Jianru Shen , Zedong Peng , Lucy Owen

A rapidly growing body of research is examining how LLMs influence developers when they code. To date, this research has tended to focus on productivity and code quality outcomes, rather than the underlying cognitive processes involved in…

Automated Program Repair (APR) agents leverage Large Language Models (LLMs) to autonomously diagnose and fix software bugs through reasoning, planning, and tool use. Despite impressive leaderboard gains on benchmarks such as SWE-bench,…

Software Engineering · Computer Science 2026-05-28 Ira Ceka , Hailie Mitchell , Saurabh Pujar , Luca Buratti , Shyam Ramji , Junfeng Yang , Gail Kaiser , Baishakhi Ray

We present the first empirical study of AI-generated pull requests that are 'silent,' meaning no comments or discussions accompany them. This absence of any comments or discussions associated with such silent AI pull requests (SPRs) poses a…

Software Engineering · Computer Science 2026-01-30 S M Mahedy Hasan , Md Fazle Rabbi , Minhaz Zibran

This paper investigates the factors influencing programmers' adoption of AI-generated JavaScript code recommendations within the context of lightweight, function-level programming tasks. It extends prior research by (1) utilizing objective…

Software Engineering · Computer Science 2026-02-18 Changwen Li , Christoph Treude , Ofir Turel

The future of software engineering--SE 3.0--is unfolding with the rise of AI teammates: autonomous, goal-driven systems collaborating with human developers. Among these, autonomous coding agents are especially transformative, now actively…

Software Engineering · Computer Science 2025-07-22 Hao Li , Haoxiang Zhang , Ahmed E. Hassan

AI agentic programming is an emerging paradigm where large language model (LLM)-based coding agents autonomously plan, execute, and interact with tools such as compilers, debuggers, and version control systems. Unlike conventional code…

Software Engineering · Computer Science 2025-09-16 Huanting Wang , Jingzhi Gong , Huawei Zhang , Jie Xu , Zheng Wang

As generative AI systems rapidly improve, a key question emerges: how do users adapt to these changes, and when does such adaptation matter for realizing performance gains? Drawing on theories of dynamic capabilities and IT complements, we…

Human-Computer Interaction · Computer Science 2026-01-08 Eaman Jahani , Benjamin S. Manning , Joe Zhang , Hong-Yi TuYe , Mohammed Alsobay , Christos Nicolaides , Siddharth Suri , David Holtz

Code review is a widespread practice to improve software quality and transfer knowledge. It is often seen as time-consuming due to the need for manual effort and potential delays. Several AI-assisted tools, such as Qodo, GitHub Copilot, and…

Despite the operational importance of hot fixes, large-scale evidence on how they reshape routine maintenance workflows, particularly in the era of autonomous coding agents, remains limited. We analyse hot fixes present in over 61,000…

Software Engineering · Computer Science 2026-04-30 Carol Hanna , Karine Even-Mendoza , W. B. Langdon , Mar Zamorano López , Justyna Petke , Federica Sarro

Code Search is a key task that many programmers often have to perform while developing solutions to problems. Current methodologies suffer from an inability to perform accurately on prompts that contain some ambiguity or ones that require…

Software Engineering · Computer Science 2024-08-22 Sarthak Jain , Aditya Dora , Ka Seng Sam , Prabhat Singh

AI is increasingly deployed in multi-agent systems; however, most research considers only the behavior of individual models. We experimentally show that multi-agent "AI organizations" are simultaneously more effective at achieving business…

Recent coding agents can generate complete codebases from simple prompts, yet existing evaluations focus on issue-level bug fixing and lag behind end-to-end development. We introduce ProjDevBench, an end-to-end benchmark that provides…

Artificial Intelligence · Computer Science 2026-02-10 Pengrui Lu , Shiqi Zhang , Yunzhong Hou , Lyumanshan Ye , Chaoyi Huang , Zixi Chen , Ji Zeng , Hantao Jiang , Pengfei Liu , Yiwei Wang , Ming-Hsuan Yang

Coding agents are increasingly deployed to autonomously maintain software, including to resolve user-reported issues: a bug report comes in and the agent creates a patch to address it. However, in any real-world deployment, they will…

Software Engineering · Computer Science 2026-05-11 Thibaud Gloaguen , Niels Mündler , Mark Müller , Veselin Raychev , Martin Vechev

Most repository-level code translation and validation techniques have been evaluated on a single source-target programming language (PL) pair, owing to the complex engineering effort required to adapt new PL pairs. Programming agents can…

Software Engineering · Computer Science 2026-04-10 Ali Reza Ibrahimzada , Brandon Paulsen , Daniel Kroening , Reyhaneh Jabbarvand