English
Related papers

Related papers: OS-Marathon: Benchmarking Computer-Use Agents on L…

200 papers

Long-term memory is crucial for agents in specialized web environments, where success depends on recalling interface affordances, state dynamics, workflows, and recurring failure modes. However, existing memory benchmarks for agents mostly…

Computation and Language · Computer Science 2026-05-13 Di Wu , Zixiang Ji , Asmi Kawatkar , Bryan Kwan , Jia-Chen Gu , Nanyun Peng , Kai-Wei Chang

Large language and vision-language models increasingly power agents that act on a user's behalf through command-line interface (CLI) harnesses. However, most agent benchmarks still rely on synthetic sandboxes, short-horizon tasks,…

End-to-end automation of realistic healthcare operations stresses three capabilities underrepresented in current benchmarks: policy density, decisions must be grounded in a large library of medical, insurance, and operational rules;…

Computer-use agent (CUA) frameworks, powered by large language models (LLMs) or multimodal LLMs (MLLMs), are rapidly maturing as assistants that can perceive context, reason, and act directly within software environments. Among their most…

Cryptography and Security · Computer Science 2025-10-13 Weidi Luo , Qiming Zhang , Tianyu Lu , Xiaogeng Liu , Bin Hu , Hung-Chun Chiu , Siyuan Ma , Yizhe Zhang , Xusheng Xiao , Yinzhi Cao , Zhen Xiang , Chaowei Xiao

Computer-use agents (CUAs) hold great promise for automating complex desktop workflows, yet progress toward general-purpose agents is bottlenecked by the scarcity of continuous, high-quality human demonstration videos. Recent work…

Machine Learning · Computer Science 2026-03-26 Xiangru Jian , Shravan Nayak , Kevin Qinghong Lin , Aarash Feizi , Kaixin Li , Patrice Bechard , Spandana Gella , Sai Rajeswar

Progress in multi-agent reinforcement learning (MARL) requires challenging benchmarks that assess the limits of current methods. However, existing benchmarks often target narrow short-horizon challenges that do not adequately stress the…

Machine Learning · Computer Science 2025-11-10 Bassel Al Omari , Michael Matthews , Alexander Rutherford , Jakob Nicolaus Foerster

We present L-MARS (Legal Multi-Agent Workflow with Orchestrated Reasoning and Agentic Search), a multi-agent retrieval framework for grounded legal question answering that decomposes queries into structured sub-problems, retrieves evidence…

Artificial Intelligence · Computer Science 2026-03-31 Ziqi Wang , Boqin Yuan

The ability of large language models (LLMs) to mimic human-like intelligence has led to a surge in LLM-based autonomous agents. Though recent LLMs seem capable of planning and reasoning given user instructions, their effectiveness in…

We present TextAtari, a benchmark for evaluating language agents on very long-horizon decision-making tasks spanning up to 100,000 steps. By translating the visual state representations of classic Atari games into rich textual descriptions,…

Computation and Language · Computer Science 2025-06-11 Wenhao Li , Wenwu Li , Chuyun Shen , Junjie Sheng , Zixiao Huang , Di Wu , Yun Hua , Wei Yin , Xiangfeng Wang , Hongyuan Zha , Bo Jin

Computer use agents are LLM-based agents that can directly interact with a graphical user interface, by processing screenshots or accessibility trees. While these systems are gaining popularity, their safety has been largely overlooked,…

Software Engineering · Computer Science 2025-10-30 Thomas Kuntz , Agatha Duzan , Hao Zhao , Francesco Croce , Zico Kolter , Nicolas Flammarion , Maksym Andriushchenko

Computer-use agents (CUAs) hold promise for automating everyday digital tasks, but their performance on long-horizon, complex problems remains unreliable. Single-rollout execution is brittle, with small errors compounding over time and…

Artificial Intelligence · Computer Science 2026-02-05 Gonzalo Gonzalez-Pumariega , Vincent Tu , Chih-Lun Lee , Jiachen Yang , Ang Li , Xin Eric Wang

Deep reasoning is fundamental for solving complex tasks, especially in vision-centric scenarios that demand sequential, multimodal understanding. However, existing benchmarks typically evaluate agents with fully synthetic, single-turn…

Large Language Models (LLMs) are increasingly used as autonomous agents in complex, long-horizon applications, where effective memory is critical for sustained performance. Yet existing memory benchmarks are largely dialogue-centric, while…

Despite the potential of language model-based agents to solve real-world tasks such as web navigation, current methods still struggle with long-horizon tasks with complex action trajectories. In contrast, humans can flexibly solve complex…

Computation and Language · Computer Science 2024-09-12 Zora Zhiruo Wang , Jiayuan Mao , Daniel Fried , Graham Neubig

Agents for computer use (ACUs) are an emerging class of systems capable of executing complex tasks on digital devices -- such as desktops, mobile phones, and web platforms -- given instructions in natural language. These agents can automate…

The development of general-purpose agents requires a shift from executing simple instructions to completing complex, real-world productivity workflows. However, current tool-use benchmarks remain misaligned with real-world requirements,…

Computation and Language · Computer Science 2026-04-20 Jize Wang , Xuanxuan Liu , Yining Li , Songyang Zhang , Yijun Wang , Zifei Shan , Xinyi Le , Cailian Chen , Xinping Guan , Dacheng Tao

Multi-agent systems based on large language models, particularly centralized architectures, have recently shown strong potential for complex and knowledge-intensive tasks. However, central agents often suffer from unstable long-horizon…

Artificial Intelligence · Computer Science 2026-01-12 Ruizhe Zhang , Xinke Jiang , Zhibang Yang , Zhixin Zhang , Jiaran Gao , Yuzhen Xiao , Hongbin Lai , Xu Chu , Junfeng Zhao , Yasha Wang

Large language model (LLM) agents are increasingly tested on complex tasks, but their ability to allocate scarce resources over long horizons remains unclear. Unlike reactive tasks with immediate feedback, this setting requires agents to…

Computer-use agents have rapidly improved on real-world tasks such as web navigation, desktop automation, and software interaction, in some cases surpassing human performance. Yet even when the task and model are unchanged, an agent that…

Artificial Intelligence · Computer Science 2026-04-21 Gonzalo Gonzalez-Pumariega , Saaket Agashe , Jiachen Yang , Ang Li , Xin Eric Wang

Processing long-form audio is a major challenge for Large Audio Language models (LALMs). These models struggle with the quadratic cost of attention ($O(N^2)$) and with modeling long-range temporal dependencies. Existing audio benchmarks are…