English
Related papers

Related papers: Programming with Pixels: Can Computer-Use Agents d…

200 papers

Large language models (LLMs) show remarkable potential to act as computer agents, enhancing human productivity and software accessibility in multi-modal tasks that require planning and reasoning. However, measuring agent performance in…

Today's world of scientific software for High Energy Physics (HEP) is powered by x86 code, while the future will be much more reliant on accelerators like GPUs and FPGAs. The portable parallelization strategies (PPS) project of the High…

A quantum computer (QC) can solve many computational problems more efficiently than a classic one. The field of QCs is growing: companies (such as DWave, IBM, Google, and Microsoft) are building QC offerings. We position that software…

Software Engineering · Computer Science 2019-07-09 Andriy Miranskyy , Lei Zhang

Most vision-language systems are static observers: they describe pixels, do not act, and cannot safely improve under shift. This passivity limits generalizable, physically grounded visual intelligence. Learning through action, not static…

Computer Vision and Pattern Recognition · Computer Science 2026-03-27 Yunpeng Zhou

Can large language model agents develop industry-level mobile applications? We introduce \textbf{SWE-Bench Mobile}, a benchmark for evaluating coding agents on realistic software engineering tasks derived from a production iOS codebase.…

Software Engineering · Computer Science 2026-02-11 Muxin Tian , Zhe Wang , Blair Yang , Zhenwei Tang , Kunlun Zhu , Honghua Dong , Hanchen Li , Xinni Xie , Guangjing Wang , Jiaxuan You

Context: Autism spectrum disorder (ASD) leads to various issues in the everyday life of autistic individuals, often resulting in unemployment and mental health problems. To improve the inclusion of autistic adults, existing studies have…

Software Engineering · Computer Science 2025-11-05 Madalena Sasportes , Grischa Liebel , Miguel Goulão

General-purpose computer-use agents have shown impressive performance across diverse digital environments. However, our new benchmark, OSExpert-Eval, indicates they remain far less helpful than human experts. Although inference-time scaling…

Artificial Intelligence · Computer Science 2026-03-10 Jiateng Liu , Zhenhailong Wang , Rushi Wang , Bingxuan Li , Jeonghwan Kim , Aditi Tiwari , Pengfei Yu , Denghui Zhang , Heng Ji

In this paper, we conduct a systematic literature review of (semi-) automated approaches to Quantum Software Engineering (QSE) and Quantum Artificial Intelligence (QAI). Prior work in the literature indicated that both Software Engineering…

Software Engineering · Computer Science 2026-04-23 Nazanin Siavash , Armin Moin

Performance optimization is a critical yet challenging aspect of software development, often requiring a deep understanding of system behavior, algorithmic tradeoffs, and careful code modifications. Although recent advances in AI coding…

Software Engineering · Computer Science 2025-12-29 Huiyun Peng , Antonio Zhong , Ricardo Andrés Calvo Méndez , Kelechi G. Kalu , James C. Davis

Large Language Models (LLM) and Generative Pre-trained Transformers (GPT), are reshaping the field of Software Engineering (SE). They enable innovative methods for executing many software engineering tasks, including automated code…

Software Engineering · Computer Science 2023-12-01 Zeeshan Rasheed , Muhammad Waseem , Kai-Kristian Kemell , Wang Xiaofeng , Anh Nguyen Duc , Kari Systä , Pekka Abrahamsson

Despite significant advancements in general-purpose AI agents, several challenges still hinder their practical application in real-world scenarios. First, the limited planning capabilities of Large Language Models (LLM) restrict AI agents…

Artificial Intelligence · Computer Science 2025-01-17 Anbang Ye , Qianran Ma , Jia Chen , Muqi Li , Tong Li , Fujiao Liu , Siqi Mai , Meichen Lu , Haitao Bao , Yang You

Several papers have recently contained reports on applying machine learning (ML) to the automation of software engineering (SE) tasks, such as project management, modeling and development. However, there appear to be no approaches comparing…

Software Engineering · Computer Science 2018-02-09 Nathalia Nascimento , Carlos Lucena , Paulo Alencar , Donald Cowan

With the rapid advancements in Large Language Models (LLMs), an increasing number of studies have leveraged LLMs as the cognitive core of agents to address complex task decision-making challenges. Specially, recent research has demonstrated…

Multiagent Systems · Computer Science 2025-03-13 Di Zhao , Longhui Ma , Siwei Wang , Miao Wang , Zhao Lv

As autonomous code agents move toward end-to-end software development, evaluating their practical autonomy becomes critical. Current benchmarks hide friction by testing agents in pre-configured environments, and their static evaluation…

Software Engineering · Computer Science 2026-05-14 Hao Guan , Lingyue Fu , Shao Zhang , Yaoming Zhu , Kangning Zhang , Lin Qiu , Xunliang Cai , Xuezhi Cao , Weiwen Liu , Weinan Zhang , Yong Yu

Students enrolled in software engineering degrees are generally required to undertake a research project in their final year through which they demonstrate the ability to conduct research, communicate outcomes, and build in-depth expertise…

Software Engineering · Computer Science 2023-02-14 Chetan Arora , Laura Tubino , Andrew Cain , Kevin Lee , Vasudha Malhotra

AI for software engineering has made remarkable progress recently, becoming a notable success within generative AI. Despite this, there are still many challenges that need to be addressed before automated software engineering reaches its…

Software Engineering · Computer Science 2025-03-31 Alex Gu , Naman Jain , Wen-Ding Li , Manish Shetty , Yijia Shao , Ziyang Li , Diyi Yang , Kevin Ellis , Koushik Sen , Armando Solar-Lezama

When assessing the quality of coding agents, predominant benchmarks focus on solving single issues on GitHub, such as SWE-Bench. In contrast, in real use, these agents solve more various and complex tasks that involve other skills such as…

Prior works on training software engineering agents have explored utilizing existing resources such as issues on GitHub repositories to construct software engineering tasks and corresponding test suites. These approaches face two key…

Software Engineering · Computer Science 2026-01-13 Yiqi Zhu , Apurva Gandhi , Graham Neubig

Agentic code assistants are a new generation of AI systems capable of performing end-to-end software engineering tasks. While these systems promise unprecedented productivity gains, their behavior and effectiveness depend heavily on…

Software Engineering · Computer Science 2026-05-26 Helio Victor F. Santos , Vitor Costa , Joao Eduardo Montandon , Marco Tulio Valente

Large vision-language models have significantly advanced GUI agents, enabling executable interaction across web, mobile, and desktop interfaces. Yet these gains largely rely on a forgiving region-tolerant paradigm, where many nearby pixels…

Artificial Intelligence · Computer Science 2026-05-18 Jingxuan Wei , Xi Bai , Shan Liu , Caijun Jia , Zheng Sun , Xinglong Xu , Siyuan Li , Linzhuang Sun , Bihui Yu , Conghui He , Cheng Tan