English
Related papers

Related papers: Computer Use at the Edge of the Statistical Precip…

200 papers

Autonomous agents have recently achieved remarkable progress across diverse domains, yet most evaluations focus on short-horizon, fully observable tasks. In contrast, many critical real-world tasks, such as large-scale software development,…

Computer-use agents hold the promise of assisting in a wide range of digital economic activities. However, current research has largely focused on short-horizon tasks over a limited set of software with limited economic value, such as basic…

Machine Learning · Computer Science 2026-04-08 Pranjal Aggarwal , Graham Neubig , Sean Welleck

AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not sufficiently difficult to meaningfully measure frontier…

Multimodal LLMs are increasingly deployed as perceptual backbones for autonomous agents in 3D environments, from robotics to virtual worlds. These applications require agents to perceive rapid state changes, attribute actions to the correct…

Computation and Language · Computer Science 2026-04-14 Yunzhe Wang , Runhui Xu , Kexin Zheng , Tianyi Zhang , Jayavibhav Niranjan Kogundi , Soham Hans , Volkan Ustun

We present AIOS 1.0, a novel platform designed to advance computer-use agent (CUA) capabilities through environmental contextualization. While existing approaches primarily focus on building more powerful agent frameworks or enhancing agent…

Artificial Intelligence · Computer Science 2025-11-04 Kai Mei , Xi Zhu , Hang Gao , Shuhang Lin , Yongfeng Zhang

Agentic systems are evaluated on benchmarks where agents interact with environments to solve tasks. Most papers report a pass@1 score computed from a single run per task, assuming this gives a reliable performance estimate. We test this…

Machine Learning · Computer Science 2026-03-26 Bjarni Haukur Bjarnason , André Silva , Martin Monperrus

Autonomous web agents solve complex browsing tasks, yet existing benchmarks measure only whether an agent finishes a task, ignoring whether it does so safely or in a way enterprises can trust. To integrate these agents into critical…

Artificial Intelligence · Computer Science 2026-03-03 Ido Levy , Ben Wiesel , Sami Marreed , Alon Oved , Avi Yaeli , Segev Shlomov

Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment. However, reward hacking, where agents maximize a score without performing the intended task, emerges…

Artificial Intelligence · Computer Science 2026-05-14 Hao Wang , Hanchen Li , Qiuyang Mang , Alvin Cheung , Koushik Sen , Dawn Song

Multimodal agents are making rapid progress on general computer-use tasks, yet existing benchmarks remain largely confined to browsers and basic desktop applications, falling short in professional software workflows that dominate real-world…

Software Engineering · Computer Science 2026-01-07 Jiaxin Ai , Yukang Feng , Fanrui Zhang , Jianwen Sun , Zizhen Li , Chuanhao Li , Yifan Chang , Wenxiao Wu , Ruoxi Wang , Mingliang Zhai , Kaipeng Zhang

Agents are rapidly advancing in automating digital work, but enterprises face a harder challenge: moving beyond prototypes to deployed systems that deliver measurable business value. This path is complicated by fragmented frameworks, slow…

Vision-Language Models (VLMs) have enabled computer use agents (CUAs) that operate GUIs autonomously, showing great potential, yet progress is limited by the lack of large-scale, open-source computer use data and foundation models. In this…

Personalized mobile agents that infer user preferences and calibrate proactive assistance hold great promise as everyday digital assistants, yet existing benchmarks fail to capture what this requires. Prior work evaluates preference…

Modern AI agents execute real-world side effects through tool calls such as file operations, shell commands, HTTP requests, and database queries. A single unsafe action, including accidental deletion, credential exposure, or data…

Artificial Intelligence · Computer Science 2026-05-07 Chenglin Yang

Recent advancements in Graphical User Interface (GUI) agents have predominantly focused on training paradigms like supervised fine-tuning (SFT) and reinforcement learning (RL). However, the challenge of high-dynamic GUI environments remains…

Computer Vision and Pattern Recognition · Computer Science 2026-05-11 Enqi Liu , Liyuan Pan , Zhi Gao , Yan Yang , Chenrui Shi , Yang Liu , Jingrong Wu , Qing Li

Computer-using agents (CUAs) are becoming increasingly capable; however, it remains difficult to scale evaluation of whether a trajectory truly fulfills a user instruction. In this work, we study reward modeling from execution video: a…

Computer Vision and Pattern Recognition · Computer Science 2026-03-12 Linxin Song , Jieyu Zhang , Huanxin Sheng , Taiwei Shi , Gupta Rahul , Yang Liu , Ranjay Krishna , Jian Kang , Jieyu Zhao

With the growing adoption of agent-based models in policy evaluation, a pressing question arises: Can such systems effectively simulate and analyze complex social scenarios to inform policy decisions? Addressing this challenge could…

Multiagent Systems · Computer Science 2025-02-13 Jiaju Kang , Puyu Han , Tian Zhang , Luqi Gong

Software robots have long been used in Robotic Process Automation (RPA) to automate mundane and repetitive computer tasks. With the advent of Large Language Models (LLMs) and their advanced reasoning capabilities, these agents are now able…

Artificial Intelligence · Computer Science 2024-12-30 Junhee Cho , Jihoon Kim , Daseul Bae , Jinho Choo , Youngjune Gwon , Yeong-Dae Kwon

With the rapid advancements in Large Language Models (LLMs), an increasing number of studies have leveraged LLMs as the cognitive core of agents to address complex task decision-making challenges. Specially, recent research has demonstrated…

Multiagent Systems · Computer Science 2025-03-13 Di Zhao , Longhui Ma , Siwei Wang , Miao Wang , Zhao Lv

Coding agents represent a new paradigm in automated software engineering, combining the reasoning capabilities of Large Language Models (LLMs) with tool-augmented interaction loops. However, coding agents still have severe limitations.…

Software Engineering · Computer Science 2026-04-06 Tural Mehtiyev , Wesley Assunção

We present HippoCamp, a new benchmark designed to evaluate agents' capabilities on multimodal file management. Unlike existing agent benchmarks that focus on tasks like web interaction, tool use, or software automation in generic settings,…

Artificial Intelligence · Computer Science 2026-04-02 Zhe Yang , Shulin Tian , Kairui Hu , Shuai Liu , Hoang-Nhat Nguyen , Yichi Zhang , Zujin Guo , Mengying Yu , Zinan Zhang , Jingkang Yang , Chen Change Loy , Ziwei Liu
‹ Prev 1 3 4 5 6 7 10 Next ›