English
Related papers

Related papers: OS-Marathon: Benchmarking Computer-Use Agents on L…

200 papers

Autonomous computer use agents that powered by multimodal large language models (MLLMs) are emerging as capable assistants for completing complex digital workflows. However, real-world execution environments are far from ideal: pop-ups,…

Artificial Intelligence · Computer Science 2026-05-26 Jingwei Sun , Jianing Zhu , Yuanyi Li , Tongliang Liu , Xia HU , Bo Han

We introduce WorkBench: a benchmark dataset for evaluating agents' ability to execute tasks in a workplace setting. WorkBench contains a sandbox environment with five databases, 26 tools, and 690 tasks. These tasks represent common business…

Computation and Language · Computer Science 2024-08-06 Olly Styles , Sam Miller , Patricio Cerda-Mardini , Tanaya Guha , Victor Sanchez , Bertie Vidgen

Computer-using agents (CUAs) must plan task workflows across diverse and evolving applications, yet progress is limited by the lack of large-scale, high-quality training data. Existing datasets are narrow, static, and costly to annotate,…

Artificial Intelligence · Computer Science 2026-03-17 Chan Hee Song , Yiwen Song , Palash Goyal , Yu Su , Oriana Riva , Hamid Palangi , Tomas Pfister

Computer Use Agents (CUAs) are designed to autonomously operate digital interfaces, yet they often fail to reliably determine whether a given task has been completed. We present an autonomous evaluation and feedback framework that uses…

Artificial Intelligence · Computer Science 2025-11-26 Marta Sumyk , Oleksandr Kosovan

In the field of MLLM-based GUI agents, compared to smartphones, the PC scenario not only features a more complex interactive environment, but also involves more intricate intra- and inter-app workflows. To address these issues, we propose a…

Computer Vision and Pattern Recognition · Computer Science 2025-02-24 Haowei Liu , Xi Zhang , Haiyang Xu , Yuyang Wanyan , Junyang Wang , Ming Yan , Ji Zhang , Chunfeng Yuan , Changsheng Xu , Weiming Hu , Fei Huang

While long-horizon agentic tasks require language agents to perform dozens of sequential decisions, training such agents with reinforcement learning remains challenging. We identify two root causes: credit misattribution, where correct…

Computation and Language · Computer Science 2026-05-08 Zixuan Wang , Yuchen Yan , Hongxing Li , Teng Pan , Dingming Li , Ruiqing Zhang , Weiming Lu , Jun Xiao , Yueting Zhuang , Yongliang Shen

Modern work relies on an assortment of digital collaboration tools, yet routine processes continue to suffer from human error and delay. To address this gap, this dissertation extends TheAgentCompany with a finance-focused environment and…

Artificial Intelligence · Computer Science 2025-12-03 Rory Milsom

Evaluating Computer Use Agents (CUAs) on interactive environments is fraught with methodological pitfalls that the field has yet to systematically address. We show that a 1MB replay script that blindly executes a recorded action sequence…

Software Engineering · Computer Science 2026-05-12 Pierluca D'Oro , Sneha Silwal , William Wong , Yuxuan Sun , Fanyi Xiao , Manchen Wang , Eric Gan , Allen Bolourchi , Joseph Tighe

Recent advances in mobile Graphical User Interface (GUI) agents highlight the growing need for comprehensive evaluation benchmarks. While new online benchmarks offer more realistic testing than offline ones, they tend to focus on the…

Computation and Language · Computer Science 2026-01-30 Qinzhuo Wu , Zhizhuo Yang , Hanhao Li , Pengzhi Gao , Wei Liu , Jian Luan

Large language models can perform well on many isolated tasks, yet they continue to struggle on multi-turn, long-horizon agentic problems that require skills such as planning, state tracking, and long context processing. In this work, we…

Artificial Intelligence · Computer Science 2026-01-26 Amin Rakhsha , Thomas Hehn , Pietro Mazzaglia , Fabio Valerio Massoli , Arash Behboodi , Tribhuvanesh Orekondy

Progress in continual reinforcement learning has been limited due to several barriers to entry: missing code, high compute requirements, and a lack of suitable benchmarks. In this work, we present CORA, a platform for Continual…

Machine Learning · Computer Science 2023-01-03 Sam Powers , Eliot Xing , Eric Kolve , Roozbeh Mottaghi , Abhinav Gupta

Long-term memory is fundamental for personalized agents capable of accumulating knowledge, reasoning over user experiences, and adapting across time. However, existing memory benchmarks primarily target declarative memory, specifically…

Computer-Use Agents (CUAs) leverage large language models to execute GUI operations on desktop environments, yet they generate actions without evaluating action quality, leading to irreversible errors that cascade through subsequent steps.…

Artificial Intelligence · Computer Science 2026-05-29 Rongqian Chen , Yu Li , Zeyu Fang , Sizhe Tang , Weidong Cao , Tian Lan

Lifelong learning is essential for intelligent agents operating in dynamic environments. Current large language model (LLM)-based agents, however, remain stateless and unable to accumulate or transfer knowledge over time. Existing…

Artificial Intelligence · Computer Science 2025-06-02 Junhao Zheng , Xidi Cai , Qiuke Li , Duzhen Zhang , ZhongZhi Li , Yingying Zhang , Le Song , Qianli Ma

Autonomous agents that navigate Graphical User Interfaces (GUIs) to automate tasks like document editing and file management can greatly enhance computer workflows. While existing research focuses on online settings, desktop environments,…

Large language models (LLMs) have shown promise as interactive agents that solve tasks through extended sequences of environment interactions. While prior work has primarily focused on system-level optimizations or algorithmic improvements,…

Artificial Intelligence · Computer Science 2026-05-05 Sunghwan Kim , Junhee Cho , Beong-woo Kwak , Taeyoon Kwon , Liang Wang , Nan Yang , Xingxing Zhang , Furu Wei , Jinyoung Yeo

Multi-agent systems (MAS) enable complex reasoning by coordinating multiple agents, but often incur high inference latency due to multi-step execution and repeated model invocations, severely limiting their scalability and usability in…

Multiagent Systems · Computer Science 2026-01-16 Xi Shi , Mengxin Zheng , Qian Lou

From professional research to everyday planning, many tasks are bottlenecked by wide-scale information seeking, which is more repetitive than cognitively complex. With the rapid development of Large Language Models (LLMs), automated search…

Computation and Language · Computer Science 2025-08-29 Ryan Wong , Jiawei Wang , Junjie Zhao , Li Chen , Yan Gao , Long Zhang , Xuan Zhou , Zuo Wang , Kai Xiang , Ge Zhang , Wenhao Huang , Yang Wang , Ke Wang

Retrieving real-time information is a fundamental capability for search-integrated agents in real-world applications. However, existing benchmarks are predominantly static and therefore fail to capture the temporal dynamics of information…

Information Retrieval · Computer Science 2026-05-12 Wenjie Zhou , Yuan Gao , Xin Zhou , Hao Fu , Zhongjian Miao , Wei Chen , Bo Chen , Xiaobing Zhao

Large language models excel at short-horizon reasoning tasks, but performance drops as reasoning horizon lengths increase. Existing approaches to combat this rely on inference-time scaffolding or costly step-level supervision, neither of…