English
Related papers

Related papers: Step-level Optimization for Efficient Computer-use…

200 papers

Autonomous graphical user interface (GUI) agents aim to facilitate task automation by interacting with the user interface without manual intervention. Recent studies have investigated eliciting the capabilities of large language models…

Computation and Language · Computer Science 2024-06-10 Zhuosheng Zhang , Aston Zhang

Interactive agent benchmarks face a tension between scalable construction and realistic workflow evaluation. Hand-authored tasks are expensive to extend and revise, while static prompt evaluation misses failures that only appear when agents…

Artificial Intelligence · Computer Science 2026-05-19 Yuxiang Lai , Peng Xia , Haonian Ji , Kaiwen Xiong , Kaide Zeng , Jiaqi Liu , Fang Wu , Jike Zhong , Zeyu Zheng , Cihang Xie , Huaxiu Yao

Computer-use agents are increasingly capable of operating on real operating systems, but this capability has also increased the risks posed by prompt injection, indirect instructions, and visual attacks. Existing defenses typically rely on…

Computation · Statistics 2026-05-14 Kebin Contreras , Carlos Hinojosa , Jorge Bacca , Bernard Ghanem

Autonomous agents that operate computers via Graphical User Interfaces (GUIs) often struggle with efficiency and reliability on complex, long-horizon tasks. While augmenting these agents with planners can improve task decomposition, they…

Computation and Language · Computer Science 2026-02-23 Linxin Song , Yutong Dai , Viraj Prabhu , Jieyu Zhang , Taiwei Shi , Li Li , Junnan Li , Silvio Savarese , Zeyuan Chen , Jieyu Zhao , Ran Xu , Caiming Xiong

The rise of personal assistant agents, e.g., OpenClaw, highlights the growing potential of large language models to support users across everyday life and work. A core challenge in these settings is proactive assistance, since users often…

Artificial Intelligence · Computer Science 2026-05-20 Haoran Zhang , Luxin Xu , Zhilin Wang , Runquan Gui , Shunkai Zhang , Haodi Lei , Zihao He , Bingsu He , Chicheng Qin , Tong Zhu , Xiaoye Qu , Yang Yang , Yu Cheng , Yafu Li

Adapting production-level computer vision tools to bespoke scientific datasets is a critical "last mile" bottleneck. Current solutions are impractical: fine-tuning requires large annotated datasets scientists often lack, while manual code…

Computer Vision and Pattern Recognition · Computer Science 2025-12-09 Xuefei , Wang , Kai A. Horstmann , Ethan Lin , Jonathan Chen , Alexander R. Farhang , Sophia Stiles , Atharva Sehgal , Jonathan Light , David Van Valen , Yisong Yue , Jennifer J. Sun

Computer-use agents(CUAs)are moving frombounded benchmarks toward real software environments, wherethey operate browsers, desktops, mobile applications, flesystems,terminals, and tool backends. In such settings, reliability isno longer…

Computation and Language · Computer Science 2026-05-11 Zejian Chen , Zhanyuan Liu , Chaozhuo Li , Mengxiang Han , Songyang Liu , Litian Zhang , Feng Gao , Yiming Hei , Xi Zhang

Evaluating GUI agents presents a distinct challenge: trajectories are long, visually grounded, and open-ended, yet evaluation must be both accurate and interpretable. Existing approaches typically apply a single holistic judgment over the…

Artificial Intelligence · Computer Science 2026-04-07 Yuwen Zhai , Runze Li , Liang Wang , Nian Shi , Liwu Xu , Wei Zhang , Ran Lin , Bo Xu , Benlei Cui

Scaling up high-quality trajectory data has long been a critical bottleneck for developing human-like computer use agents. We introduce PC Agent-E, an efficient agent training framework that significantly reduces reliance on large-scale…

Artificial Intelligence · Computer Science 2026-03-04 Yanheng He , Jiahe Jin , Pengfei Liu

Recent advances in frontier large language models have enabled code review agents that operate in open-ended, reasoning-intensive settings. However, the lack of standardized benchmarks and granular evaluation protocols makes it difficult to…

Software Engineering · Computer Science 2026-03-13 Kristen Pereira , Neelabh Sinha , Rajat Ghosh , Debojyoti Dutta

In high-performance computing (HPC) environments, system monitoring data is often unlabeled and high-dimensional, making it difficult to reliably detect and understand anomalous computing nodes. The growing scale and dimensionality of the…

Distributed, Parallel, and Cluster Computing · Computer Science 2026-04-15 Allison Austin , Shilpika , Yan To Linus Lam , Yun-Hsin Kuo , Venkatram Vishwanath , Michael E. Papka , Kwan-Liu Ma

Large language model (LLM) agents execute tasks through multi-step workflows that combine planning, memory, and tool use. While this design enables autonomy, it also expands the attack surface for backdoor threats. Backdoor triggers…

Artificial Intelligence · Computer Science 2026-01-13 Yunhao Feng , Yige Li , Yutao Wu , Yingshui Tan , Yanming Guo , Yifan Ding , Kun Zhai , Xingjun Ma , Yu-Gang Jiang

Graphical User Interface (GUI) agents possess significant commercial and social value, and GUI agents powered by advanced multimodal large language models (MLLMs) have demonstrated remarkable potential. Currently, existing GUI agents…

Artificial Intelligence · Computer Science 2025-09-05 Weizhi Chen , Ziwei Wang , Leyang Yang , Sheng Zhou , Xiaoxuan Tang , Jiajun Bu , Yong Li , Wei Jiang

A stochastic Forward-Backward algorithm with a constant step is studied. At each time step, this algorithm involves an independent copy of a couple of random maximal monotone operators. Defining a mean operator as a selection integral, the…

Optimization and Control · Mathematics 2018-04-05 Pascal Bianchi , Walid Hachem , Adil Salim

Computer-Use Agents (CUAs) are an increasingly deployed class of agents that take actions on GUIs to accomplish user goals. In this paper, we show that CUAs consistently exhibit Blind Goal-Directedness (BGD): a bias to pursue goals…

Large language models and AI agents have recently shown promise in automating software performance optimization, but existing approaches predominantly rely on local, syntax-driven code transformations. This limits their ability to reason…

Software Engineering · Computer Science 2026-03-17 Huiyun Peng , Parth Vinod Patil , Antonio Zhong Qiu , George K. Thiruvathukal , James C. Davis

As Large Language Models increasingly automate complex, long-horizon tasks such as \emph{vibe coding}, a supervision gap has emerged. While models excel at execution, users often struggle to guide them effectively due to insufficient domain…

Artificial Intelligence · Computer Science 2026-02-09 Enyu Zhou , Zhiheng Xi , Long Ma , Zhihao Zhang , Shihan Dou , Zhikai Lei , Guoteng Wang , Rui Zheng , Hang Yan , Tao Gui , Qi Zhang , Xuanjing Huang

Autonomous agents that accomplish complex computer tasks with minimal human interventions have the potential to transform human-computer interaction, significantly enhancing accessibility and productivity. However, existing benchmarks…

We consider the problem of using an autonomous agent to persistently monitor a collection of dynamic targets distributed in an environment. We generalize existing work by allowing the agent's dynamics to vary throughout the environment,…

Optimization and Control · Mathematics 2025-03-03 Jonas Hall , Christos G. Cassandras , Sean B. Andersson

Test-time scaling has become a standard way to improve performance and boost reliability of neural network models. However, its behavior on agentic, multi-step tasks remains less well-understood: small per-step errors can compound over long…

Artificial Intelligence · Computer Science 2026-02-13 Nicholas Lee , Lutfi Eren Erdogan , Chris Joseph John , Surya Krishnapillai , Michael W. Mahoney , Kurt Keutzer , Amir Gholami