English
Related papers

Related papers: A Task-Level Evaluation of AI Agents in Open-Sourc…

200 papers

Code-generating Artificial Intelligence has gained popularity within both professional and educational programming settings over the past several years. While research and pedagogy are beginning to cope with this change, computing students…

Human-Computer Interaction · Computer Science 2026-04-21 Nicholas Gardella , James Prather , Juho Leinonen , Paul Denny , Raymond Pettit , Sara L. Riggs

The landscape of software development has witnessed a paradigm shift with the advent of AI-powered assistants, exemplified by GitHub Copilot. However, existing solutions are not leveraging all the potential capabilities available in an IDE…

Software Engineering · Computer Science 2024-03-14 Michele Tufano , Anisha Agarwal , Jinu Jang , Roshanak Zilouchian Moghaddam , Neel Sundaresan

This mixed-methods study examined computational thinking (CT) development among 93 pre-high school students in a five-day AI agent creation workshop using CocoFlow, a no-code platform. Integrating pre-post assessments, behavioral logs, and…

Computers and Society · Computer Science 2026-05-15 Yimeng Sun , Haiyang Xin , Qiannan Niu , Shuang Li , Lingyun Huang , Gaowei Chen

Frontier coding agents are increasingly used in workflows where users supervise progress primarily through repeated improvement of a public score, namely the reported score on a public evaluation file with labels in the workspace, rather…

Computation and Language · Computer Science 2026-04-23 Hardy Chen , Nancy Lau , Haoqin Tu , Shuo Yan , Xiangyan Liu , Zijun Wang , Juncheng Wu , Michael Qizhe Shieh , Alvaro A. Cardenas , Cihang Xie , Yuyin Zhou

Conversational agents have been gaining increasing popularity in recent years. Influenced by the widespread adoption of task-oriented agents such as Apple Siri and Amazon Alexa, these agents are being deployed into various applications to…

Human-Computer Interaction · Computer Science 2024-01-17 Christopher Clarke , Karthik Krishnamurthy , Walter Talamonti , Yiping Kang , Lingjia Tang , Jason Mars

Involving humans directly for the benefit of AI agents' training is getting traction thanks to several advances in reinforcement learning and human-in-the-loop learning. Humans can provide rewards to the agent, demonstrate tasks, design a…

Artificial Intelligence · Computer Science 2021-06-23 AI Redefined , Sai Krishna Gottipati , Sagar Kurandwad , Clodéric Mars , Gregory Szriftgiser , François Chabot

We present a two-stage pipeline for AI-assisted improvement of published algorithm implementations. In the first stage, a large language model with research capabilities identifies recently published algorithms satisfying explicit…

Software Engineering · Computer Science 2026-04-16 Worasait Suwannik

The rise of AI agents is transforming how software can be built. The promise of agents is that developers might write code quicker, delegate multiple tasks to different agents, and even write a full piece of software purely out of natural…

Software Engineering · Computer Science 2025-12-17 Ruanqianqian Huang , Avery Reyna , Sorin Lerner , Haijun Xia , Brian Hempel

Despite long-standing efforts in accelerating scientific discovery with AI, building AI co-scientists remains challenging due to limited high-quality data for training and evaluation. To tackle this data scarcity issue, we present AutoSDT,…

Pull-based development has enabled numerous volunteers to contribute to open-source projects with fewer barriers. Nevertheless, a considerable amount of pull requests (PRs) with valid contributions are abandoned by their contributors,…

Software Engineering · Computer Science 2024-08-23 SayedHassan Khatoonabadi , Diego Elias Costa , Rabe Abdalkareem , Emad Shihab

Agentic Coding, powered by autonomous agents such as GitHub Copilot and Cursor, enables developers to generate code, tests, and pull requests from natural language instructions alone. While this accelerates implementation, it produces…

Software Engineering · Computer Science 2026-02-20 Kan Watanabe , Tatsuya Shirai , Yutaro Kashiwa , Hajimu Iida

Large Language Models (LLMs) based autonomous agents demonstrate multifaceted capabilities to contribute substantially to economic production. However, existing benchmarks remain focused on single agentic capability, failing to capture…

Artificial Intelligence · Computer Science 2026-04-24 Keyu Li , Junhao Shi , Yang Xiao , Mohan Jiang , Jie Sun , Yunze Wu , Dayuan Fu , Shijie Xia , Xiaojie Cai , Tianze Xu , Weiye Si , Wenjie Li , Dequan Wang , Pengfei Liu

As full AI-based automation remains out of reach in most real-world applications, the focus has instead shifted to leveraging the strengths of both human and AI agents, creating effective collaborative systems. The rapid advances in this…

Human-Computer Interaction · Computer Science 2024-04-19 Steffen Holter , Mennatallah El-Assady

This paper presents the first large-scale field study of the adoption, usage intensity, and use cases of general-purpose AI agents operating in open-world web environments. Our analysis centers on Comet, an AI-powered browser developed by…

Machine Learning · Computer Science 2025-12-11 Jeremy Yang , Noah Yonack , Kate Zyskowski , Denis Yarats , Johnny Ho , Jerry Ma

We present Code Broker, a multi agent system built on Google s Agent Development Kit ADK that analyses Python source code from individual files, local directory trees, or remote GitHub repositories and generates structured, actionable…

Software Engineering · Computer Science 2026-05-07 Samer Attrah

While AI excels at generating text, audio, images, and videos, creating interactive audio-visual content such as video games remains challenging. Current LLMs can generate JavaScript games and animations, but lack automated evaluation…

Artificial Intelligence · Computer Science 2025-08-04 Alexia Jolicoeur-Martineau

AI coding assistants have transformed software development, raising questions about transparency and attribution practices. We examine the "AI attribution paradox": how developers strategically balance acknowledging AI assistance with…

Software Engineering · Computer Science 2025-12-02 Obada Kraishan

In recent years, AI-based software engineering has progressed from pre-trained models to advanced agentic workflows, with Software Development Agents representing the next major leap. These agents, capable of reasoning, planning, and…

Software Engineering · Computer Science 2024-12-30 Zhi Chen , Lingxiao Jiang

Cross-language migration of large software systems is a persistent engineering challenge, particularly when the source codebase evolves rapidly. We present a methodology for LLM-assisted continuous code translation in which a large language…

Software Engineering · Computer Science 2026-04-14 Jinhua Wang , Biswa Sengupta

Explainable AI techniques that describe agent reward functions can enhance human-robot collaboration in a variety of settings. One context where human understanding of agent reward functions is particularly beneficial is in the value…

Robotics · Computer Science 2021-10-11 Lindsay Sanneman , Julie Shah