English
Related papers

Related papers: Gym-Anything: Turn any Software into an Agent Envi…

200 papers

Agents are rapidly advancing in automating digital work, but enterprises face a harder challenge: moving beyond prototypes to deployed systems that deliver measurable business value. This path is complicated by fragmented frameworks, slow…

Artificial intelligence (AI) has become a powerful tool for economic research, enabling large-scale simulation and policy optimization. However, applying AI effectively requires simulation platforms for scalable training and evaluation-yet…

General Economics · Economics 2025-06-17 Qirui Mi , Qipeng Yang , Zijun Fan , Wentian Fan , Heyang Ma , Chengdong Ma , Siyu Xia , Bo An , Jun Wang , Haifeng Zhang

Deep reasoning is fundamental for solving complex tasks, especially in vision-centric scenarios that demand sequential, multimodal understanding. However, existing benchmarks typically evaluate agents with fully synthetic, single-turn…

Real-world digital environments are highly diverse and dynamic. These characteristics cause agents to frequently encounter unseen environments and distribution shifts, making continual learning in such environments essential for…

Computation and Language · Computer Science 2026-05-12 Tianci Xue , Zeyi Liao , Tianneng Shi , Zilu Wang , Kai Zhang , Dawn Song , Yu Su , Huan Sun

Graphical User Interfaces (GUIs) are central to human-computer interaction, yet automating complex GUI tasks remains a major challenge for autonomous agents, largely due to a lack of scalable, high-quality training data. While recordings of…

Computer Vision and Pattern Recognition · Computer Science 2026-01-13 Yichun Zhang , Xiangwu Guo , Yauhong Goh , Jessica Hu , Zhiheng Chen , Xin Wang , Difei Gao , Mike Zheng Shou

Scientific reasoning inherently demands integrating sophisticated toolkits to navigate domain-specific knowledge. Yet, current benchmarks largely overlook agents' ability to orchestrate tools for such rigorous workflows. To bridge this gap,…

Claw-style environments support multi-step workflows over local files, tools, and persistent workspace states. However, scalable development around these environments remains constrained by the absence of a systematic framework, especially…

Computation and Language · Computer Science 2026-05-19 Fei Bai , Huatong Song , Shuang Sun , Daixuan Cheng , Yike Yang , Chuan Hao , Renyuan Li , Feng Chang , Yuan Wei , Ran Tao , Bryan Dai , Jian Yang , Wayne Xin Zhao , Ji-Rong Wen

Computer-use agents (CUAs) hold promise for automating everyday digital tasks, but their performance on long-horizon, complex problems remains unreliable. Single-rollout execution is brittle, with small errors compounding over time and…

Artificial Intelligence · Computer Science 2026-02-05 Gonzalo Gonzalez-Pumariega , Vincent Tu , Chih-Lun Lee , Jiachen Yang , Ang Li , Xin Eric Wang

Memory is a central capability for LLM agents operating across long-horizon tasks. Existing memory benchmarks predominantly evaluate retention of personalized information in multi-turn chat scenarios, overlooking the dynamic memory…

Computation and Language · Computer Science 2026-05-21 Wujiang Xu , Yu Wang , Kai Mei , Kaiqu Liang , Zhenting Wang , Mingyu Jin , Han Zhang , Shi-Xiong Zhang , Wenyue Hua , Sambit Sahu , Dimitris N. Metaxas

Repurposing large vision-language models (LVLMs) as computer use agents (CUAs) has led to substantial breakthroughs, primarily driven by human-labeled data. However, these models often struggle with novel and specialized software,…

Artificial Intelligence · Computer Science 2025-08-13 Zeyi Sun , Ziyu Liu , Yuhang Zang , Yuhang Cao , Xiaoyi Dong , Tong Wu , Dahua Lin , Jiaqi Wang

Agent-based models (ABMs) and video games, including those taking advantage of virtual reality (VR), have undergone a remarkable parallel evolution, achieving impressive levels of complexity and sophistication. This paper argues that while…

Recent progress in GUI agents has substantially improved visual grounding, yet robust planning remains challenging, particularly when the environment deviates from a canonical initial state. In real applications, users often invoke…

Artificial Intelligence · Computer Science 2026-05-26 Henry Hengyuan Zhao , Kaiming Yang , Wendi Yu , Difei Gao , Mike Zheng Shou

Earth Observation (EO) analysis is inherently interactive: resolving uncertainty often requires expanding the region of interest, retrieving historical observations, and switching across sensors such as optical and Synthetic Aperture Radar.…

Artificial Intelligence · Computer Science 2026-05-05 Sai Ma , Zhuang Li , Sichao Li , Xinyue Xu , Ruibiao Zhu , Tony Boston , John A. Taylor

Computer-use agents(CUAs)are moving frombounded benchmarks toward real software environments, wherethey operate browsers, desktops, mobile applications, flesystems,terminals, and tool backends. In such settings, reliability isno longer…

Computation and Language · Computer Science 2026-05-11 Zejian Chen , Zhanyuan Liu , Chaozhuo Li , Mengxiang Han , Songyang Liu , Litian Zhang , Feng Gao , Yiming Hei , Xi Zhang

Generative AI is being leveraged to solve a variety of computer-use tasks involving desktop applications. State-of-the-art systems have focused solely on improving accuracy on leading benchmarks. However, these systems are practically…

Artificial Intelligence · Computer Science 2026-05-19 Reyna Abhyankar , Qi Qi , Yiying Zhang

For decades, human-computer interaction has fundamentally been manual. Even today, almost all productive work done on the computer necessitates human input at every step. Autonomous virtual agents represent an exciting step in automating…

Artificial Intelligence · Computer Science 2024-07-23 Raghav Kapoor , Yash Parag Butala , Melisa Russak , Jing Yu Koh , Kiran Kamble , Waseem Alshikh , Ruslan Salakhutdinov

Large Language Models (LLMs) are increasingly relied upon for coding tasks, yet in most scenarios it is assumed that all relevant information can be either accessed in context or matches their training data. We posit that LLMs can benefit…

In the pursuit of artificial general intelligence, our most significant measurement of progress is an agent's ability to achieve goals in a wide range of environments. Existing platforms for constructing such environments are typically…

Benchmarks for large multimodal language models (MLMs) now serve to simultaneously assess the general capabilities of models instead of evaluating for a specific capability. As a result, when a developer wants to identify which models to…

Computer Vision and Pattern Recognition · Computer Science 2025-01-28 Jieyu Zhang , Weikai Huang , Zixian Ma , Oscar Michel , Dong He , Tanmay Gupta , Wei-Chiu Ma , Ali Farhadi , Aniruddha Kembhavi , Ranjay Krishna

Tool use has turned large language models (LLMs) into powerful agents that can perform complex multi-step tasks by dynamically utilising external software components. However, these tools must be implemented in advance by human developers,…

Computation and Language · Computer Science 2025-06-02 Georg Wölflein , Dyke Ferber , Daniel Truhn , Ognjen Arandjelović , Jakob Nikolas Kather