English
Related papers

Related papers: WebCanvas: Benchmarking Web Agents in Online Envir…

200 papers

AI agents may be able to automate your inbox, but can they automate other routine aspects of your life? Everyday online tasks offer a realistic yet unsolved testbed for evaluating the next generation of AI agents. To this end, we introduce…

LLM-based web agents have the potential to automate long-running web tasks, such as searching for products in multiple e-shops and subsequently ordering the cheapest products that meet the users needs. Benchmarks for evaluating web agents…

Computation and Language · Computer Science 2026-05-01 Ralph Peeters , Aaron Steiner , Luca Schwarz , Julian Yuya Caspary , Christian Bizer

General web-based agents are increasingly essential for interacting with complex web environments, yet their performance in real-world web applications remains poor, yielding extremely low accuracy even with state-of-the-art frontier…

Artificial Intelligence · Computer Science 2026-03-03 Segev Shlomov , Ben wiesel , Aviad Sela , Ido Levy , Liane Galanti , Roy Abitbol

Recent progress in GUI agents has substantially improved visual grounding, yet robust planning remains challenging, particularly when the environment deviates from a canonical initial state. In real applications, users often invoke…

Artificial Intelligence · Computer Science 2026-05-26 Henry Hengyuan Zhao , Kaiming Yang , Wendi Yu , Difei Gao , Mike Zheng Shou

Large language models (LLMs) show remarkable potential to act as computer agents, enhancing human productivity and software accessibility in multi-modal tasks that require planning and reasoning. However, measuring agent performance in…

Existing web agent benchmarks have largely converged on short, single-site tasks that frontier models are approaching saturation on. However, real world web use consists of long-horizon, multi-site workflows. Common web navigation tasks,…

Machine Learning · Computer Science 2026-04-29 Lawrence Keunho Jang , Jing Yu Koh , Daniel Fried , Ruslan Salakhutdinov

Autonomous agents capable of planning, reasoning, and executing actions on the web offer a promising avenue for automating computer tasks. However, the majority of existing benchmarks primarily focus on text-based agents, neglecting many…

Autonomous embodied agents live on an Internet of multimedia websites. Can they hop around multimodal websites to complete complex user tasks? Existing benchmarks fail to assess them in a realistic, evolving environment for their embodiment…

Computer Vision and Pattern Recognition · Computer Science 2025-07-01 Shulin Tian , Ziniu Zhang , Liangyu Chen , Ziwei Liu

The improvement of web agents on current benchmarks raises the question: Do today's agents perform just as well when the web changes? We introduce TimeWarp, a benchmark that emulates the evolving web using containerized environments that…

Artificial Intelligence · Computer Science 2026-03-06 Md Farhan Ishmam , Kenneth Marino

The rapid advancement of multimodal large language models has enabled agents to operate mobile devices by directly interacting with graphical user interfaces, opening new possibilities for mobile automation. However, real-world mobile tasks…

Artificial Intelligence · Computer Science 2025-10-17 Yuanyi Song , Heyuan Huang , Qiqiang Lin , Yin Zhao , Xiangmou Qu , Jun Wang , Xingyu Lou , Weiwen Liu , Zhuosheng Zhang , Jun Wang , Yong Yu , Weinan Zhang , Zhaoxiang Wang

Workspace learning requires AI agents to identify, reason over, exploit, and update explicit and implicit dependencies among heterogeneous files in a worker's workspace, enabling them to complete both routine and advanced tasks effectively.…

Despite advances in multimodal large language models, autonomous web agents still struggle to reliably execute long-horizon tasks on complex and dynamic web interfaces. Existing agents often suffer from inaccurate element grounding, the…

Artificial Intelligence · Computer Science 2026-02-03 Aiden Yiliu Li , Xinyue Hao , Shilong Liu , Mengdi Wang

Web agents have shown great promise in performing many tasks on ecommerce website. To assess their capabilities, several benchmarks have been introduced. However, current benchmarks in the e-commerce domain face two major problems. First,…

Computation and Language · Computer Science 2026-04-22 Xianren Zhang , Shreyas Prasad , Di Wang , Qiuhai Zeng , Suhang Wang , Wenbo Yan , Mat Hans

Can advanced multi-modal models effectively tackle complex web-based tasks? Such tasks are often found on crowdsourcing platforms, where crowdworkers engage in challenging micro-tasks within web-based environments. Building on this idea, we…

Artificial Intelligence · Computer Science 2025-02-25 Kevin Xu , Yeganeh Kordi , Tanay Nayak , Adi Asija , Yizhong Wang , Kate Sanders , Adam Byerly , Jingyu Zhang , Benjamin Van Durme , Daniel Khashabi

Language-model agents are increasingly used as persistent coworkers that assist users across multiple working days. During such workflows, the surrounding environment may change independently of the agent: new emails arrive, calendar…

Web agents based on large language models have demonstrated promising capability in automating web tasks. However, current web agents struggle to reason out sensible actions due to the limitations of predicting environment changes, and…

Artificial Intelligence · Computer Science 2026-02-18 Zhouzhou Shen , Xueyu Hu , Xiyun Li , Tianqing Fang , Juncheng Li , Shengyu Zhang

Among existing online mobile-use benchmarks, AndroidWorld has emerged as the dominant benchmark due to its reproducible environment and deterministic evaluation; however, recent agents achieving over 90% success rates indicate its…

Computation and Language · Computer Science 2026-01-01 Quyu Kong , Xu Zhang , Zhenyu Yang , Nolan Gao , Chen Liu , Panrong Tong , Chenglin Cai , Hanzhang Zhou , Jianan Zhang , Liangyu Chen , Zhidan Liu , Steven Hoi , Yue Wang

Multimodal agents are making rapid progress on general computer-use tasks, yet existing benchmarks remain largely confined to browsers and basic desktop applications, falling short in professional software workflows that dominate real-world…

Software Engineering · Computer Science 2026-01-07 Jiaxin Ai , Yukang Feng , Fanrui Zhang , Jianwen Sun , Zizhen Li , Chuanhao Li , Yifan Chang , Wenxiao Wu , Ruoxi Wang , Mingliang Zhai , Kaipeng Zhang

The emergence of "vibe coding" platforms, where users describe applications in natural language and AI agents autonomously generate full-stack software, has created a need for rigorous evaluation beyond code-level benchmarks. In order to…

Multiagent Systems · Computer Science 2026-05-07 Siddhant Saxena , Nilesh Trivedi , Vinayaka Jyothi

Current evaluation of web agents largely reduces to binary success metrics or conformity to a single reference trajectory, ignoring the structural diversity present in benchmark datasets. We present WebGraphEval, a framework that abstracts…

Artificial Intelligence · Computer Science 2025-10-23 Yaoyao Qian , Yuanli Wang , Jinda Zhang , Yun Zong , Meixu Chen , Hanhan Zhou , Jindan Huang , Yifan Zeng , Xinyu Hu , Chan Hee Song , Danqing Zhang