English
Related papers

Related papers: Programming with Pixels: Can Computer-Use Agents d…

200 papers

Autonomous AI agents are transforming software development and redefining how developers collaborate with AI. Prior research shows that the adoption and use of AI-powered tools differ between core and peripheral developers. However, it…

Software Engineering · Computer Science 2026-01-29 Shamse Tasnim Cynthia , Joy Krishan Das , Banani Roy

We introduce SWE Atlas, a benchmark suite for coding agents spanning three professional software engineering workflows: Codebase Q&A (124 tasks), Test Writing (90 tasks), and Refactoring (70 tasks). SWE Atlas differs from prior SWE…

Agent skills, structured procedural knowledge packages injected at inference time, are increasingly used to augment LLM agents on software engineering tasks. However, their real utility in end-to-end development settings remains unclear. We…

Software Engineering · Computer Science 2026-03-17 Tingxu Han , Yi Zhang , Wei Song , Chunrong Fang , Zhenyu Chen , Youcheng Sun , Lijie Hu

Over-reliance on AI systems can undermine users' critical thinking and promote complacency, a risk intensified by the emergence of agentic AI systems that operate with minimal human involvement. In software engineering, agentic coding…

Human-Computer Interaction · Computer Science 2026-03-19 Carlos Rafael Catalan , Lheane Marie Dizon , Patricia Nicole Monderin , Emily Kuang

The rise of AI agents is transforming how software can be built. The promise of agents is that developers might write code quicker, delegate multiple tasks to different agents, and even write a full piece of software purely out of natural…

Software Engineering · Computer Science 2025-12-17 Ruanqianqian Huang , Avery Reyna , Sorin Lerner , Haijun Xia , Brian Hempel

Computer-use agents (CUAs) hold promise for automating everyday digital tasks, but their performance on long-horizon, complex problems remains unreliable. Single-rollout execution is brittle, with small errors compounding over time and…

Artificial Intelligence · Computer Science 2026-02-05 Gonzalo Gonzalez-Pumariega , Vincent Tu , Chih-Lun Lee , Jiachen Yang , Ang Li , Xin Eric Wang

We present Agent S, an open agentic framework that enables autonomous interaction with computers through a Graphical User Interface (GUI), aimed at transforming human-computer interaction by automating complex, multi-step tasks. Agent S…

Artificial Intelligence · Computer Science 2024-10-11 Saaket Agashe , Jiuzhou Han , Shuyu Gan , Jiachen Yang , Ang Li , Xin Eric Wang

The growth of Large Language Model (LLM) technology has raised expectations for automated coding. However, software engineering is more than coding and is concerned with activities including maintenance and evolution of a project. In this…

Software Engineering · Computer Science 2025-12-09 Leonhard Applis , Yuntong Zhang , Shanchao Liang , Nan Jiang , Lin Tan , Abhik Roychoudhury

Proactive agents that anticipate user needs and autonomously execute tasks hold great promise as digital assistants, yet the lack of realistic user simulation frameworks hinders their development. Existing approaches model apps as flat…

Artificial Intelligence · Computer Science 2026-04-02 Deepak Nathani , Cheng Zhang , Chang Huan , Jiaming Shan , Yinfei Yang , Alkesh Patel , Zhe Gan , William Yang Wang , Michael Saxon , Xin Eric Wang

In immersive augmented reality (IAR), users can wear a head-mounted display to see computer-generated images superimposed to their view of the world. IAR was shown to be beneficial across several domains, e.g., automotive, medicine, gaming…

Software Engineering · Computer Science 2020-01-07 Leonel Merino , Mircea Lungu , Christoph Seidl

In high performance domains like image processing, physics simulation or machine learning, program performance is critical. Programmers called performance engineers are responsible for the challenging task of optimising programs. Two major…

Programming Languages · Computer Science 2022-12-26 Thomas Koehler

Web applications are prime targets for cyberattacks as gateways to critical services and sensitive data. Traditional penetration testing is costly and expertise-intensive, making it difficult to scale with the growing web ecosystem. While…

Cryptography and Security · Computer Science 2025-10-15 Xiaoxue Ren , Penghao Jiang , Kaixin Li , Zhiyong Huang , Xiaoning Du , Jiaojiao Jiang , Zhenchang Xing , Jiamou Sun , Terry Yue Zhuo

In recent years, AI-based software engineering has progressed from pre-trained models to advanced agentic workflows, with Software Development Agents representing the next major leap. These agents, capable of reasoning, planning, and…

Software Engineering · Computer Science 2024-12-30 Zhi Chen , Lingxiao Jiang

AI coding agents demonstrate strong performance on general-purpose software benchmarks. However, their ability to handle 5G network engineering tasks remains unexplored. We propose SWE-Bench~5G, the first benchmark designed to investigate…

Networking and Internet Architecture · Computer Science 2026-04-30 Jiao Chen , Jianhua Tang , Xiaotong Yang , Zuohong Lv

AI agents have recently shown significant promise in software engineering. Much public attention has been transfixed on the topic of code generation from Large Language Models (LLMs) via a prompt. However, software engineering is much more…

Software Engineering · Computer Science 2025-09-23 Abhik Roychoudhury

Physics Engines (PEs) are fundamental software frameworks that simulate physical interactions in applications ranging from entertainment to safety-critical systems. Despite their importance, PEs suffer from physics failures, deviations from…

Computer Vision and Pattern Recognition · Computer Science 2025-07-31 Shuqing Li , Qiang Chen , Xiaoxue Ren , Michael R. Lyu

Computer-using agents have shown strong potential to boost human productivity and enable new application forms across platforms. While recent advances have led to usable applications, existing benchmarks fail to account for the internal…

After years of using Graphics Processing Units (GPUs) to accelerate scientific applications in fields as varied as tomography, computer vision, climate modeling, digital forensics, geospatial databases, particle physics, radio astronomy,…

Software Engineering · Computer Science 2020-05-28 Ben van Werkhoven , Willem Jan Palenstijn , Alessio Sclocco

Agents operating in complex software environments benefit from reasoning about the consequences of their actions, as even a single incorrect user interface (UI) operation can derail long, artifact-preserving workflows. This challenge is…

Computer-use agents face a fundamental limitation. They rely exclusively on primitive GUI actions (click, type, scroll), creating brittle execution chains prone to cascading failures. While API-driven agents harness rich capabilities…

Computer Vision and Pattern Recognition · Computer Science 2026-05-27 Yuhao Yang , Zhen Yang , Zi-Yi Dou , Anh Nguyen , Keen You , Omar Attia , Andrew Szot , Michael Feng , Ram Ramrakhya , Alexander Toshev , Chao Huang , Yinfei Yang , Zhe Gan