English
Related papers

Related papers: TerminalWorld: Benchmarking Agents on Real-World T…

200 papers

Recent advances in mobile Graphical User Interface (GUI) agents highlight the growing need for comprehensive evaluation benchmarks. While new online benchmarks offer more realistic testing than offline ones, they tend to focus on the…

Computation and Language · Computer Science 2026-01-30 Qinzhuo Wu , Zhizhuo Yang , Hanhao Li , Pengzhi Gao , Wei Liu , Jian Luan

Autonomous data analysis agents are increasingly expected to conduct exploratory analysis with limited human guidance about data. However, existing benchmarks typically evaluate such agents in prior-guided settings, providing selected data…

Artificial Intelligence · Computer Science 2026-05-28 Qiaohong Zhang , Weihao Ye , Jialong Chen , Yi Luo , BoYuan Li , Bowen Deng , Zibin Zheng , Jianhao Lin , Wei-Shi Zheng , Chuan Chen

Web agents require massive trajectories to generalize, yet real-world training is constrained by network latency, rate limits, and safety risks. We introduce \textbf{WebWorld} series, the first open-web simulator trained at scale. While…

Artificial Intelligence · Computer Science 2026-02-17 Zikai Xiao , Jianhong Tu , Chuhang Zou , Yuxin Zuo , Zhi Li , Peng Wang , Bowen Yu , Fei Huang , Junyang Lin , Zuozhu Liu

We present OpenComputer, a verifier-grounded framework for constructing verifiable software worlds for computer-use agents. OpenComputer integrates four components: (1) app-specific state verifiers that expose structured inspection…

Artificial Intelligence · Computer Science 2026-05-20 Jinbiao Wei , Qianran Ma , Yilun Zhao , Xiao Zhou , Kangqi Ni , Guo Gan , Arman Cohan

Recent progress in GUI agents has substantially improved visual grounding, yet robust planning remains challenging, particularly when the environment deviates from a canonical initial state. In real applications, users often invoke…

Artificial Intelligence · Computer Science 2026-05-26 Henry Hengyuan Zhao , Kaiming Yang , Wendi Yu , Difei Gao , Mike Zheng Shou

Current LLM agent benchmarks, which predominantly focus on binary pass/fail tasks such as code generation or search-based question answering, often neglect the value of real-world engineering that is often captured through the iterative…

As multimodal large language models (MLLMs) advance, MLLM-based virtual agents have demonstrated remarkable performance. However, existing benchmarks face significant limitations, including uncontrollable task complexity, extensive manual…

Computer Vision and Pattern Recognition · Computer Science 2025-06-11 Wendong Bu , Yang Wu , Qifan Yu , Minghe Gao , Bingchen Miao , Zhenkui Zhang , Kaihang Pan , Yunfei Li , Mengze Li , Wei Ji , Juncheng Li , Siliang Tang , Yueting Zhuang

Mastering terminal environments requires language agents capable of multi-step planning, feedback-grounded execution, and dynamic state adaptation. However, training such agents is currently bottlenecked by a reliance on scraped external…

Computation and Language · Computer Science 2026-05-29 Xiaoxuan Peng , Kaiqi Zhang , Xinyu Lu , Boxi Cao , Yaojie Lu , Hongyu Lin , Xianpei Han , Le Sun

Mobile agents can autonomously complete user-assigned tasks through GUI interactions. However, existing mainstream evaluation benchmarks, such as AndroidWorld, operate by connecting to a system-level Android emulator and provide evaluation…

Artificial Intelligence · Computer Science 2026-04-14 Yunfei Feng , Xi Zhao , Cheng Zhang , Dahu Feng , Daolin Cheng , Jianqi Yu , Yubin Xia , Erhu Feng

LLM agents are expected to complete end-to-end units of work across software tools, business services, and local workspaces. Yet many agent benchmarks freeze a curated task set at release time and grade mainly the final response, making it…

Software Engineering · Computer Science 2026-05-04 Chenxin Li , Zhengyang Tang , Mingxin Huang , Yunlong Lin , Shijue Huang , Shengyuan Liu , Bowen Ye , Rang Li , Lei Li , Benyou Wang , Yixuan Yuan

In this paper, we introduce OSUniverse: a benchmark of complex, multimodal desktop-oriented tasks for advanced GUI-navigation AI agents that focuses on ease of use, extensibility, comprehensive coverage of test cases, and automated…

Artificial Intelligence · Computer Science 2025-05-07 Mariya Davydova , Daniel Jeffries , Patrick Barker , Arturo Márquez Flores , Sinéad Ryan

We introduce DABstep, a novel benchmark for evaluating AI agents on realistic multi-step data analysis tasks. DABstep comprises over 450 real-world challenges derived from a financial analytics platform, requiring models to combine…

Machine Learning · Computer Science 2025-07-01 Alex Egg , Martin Iglesias Goyanes , Friso Kingma , Andreu Mora , Leandro von Werra , Thomas Wolf

Penetration testing is essential for assessing and strengthening system security against real-world threats, yet traditional workflows remain highly manual, expertise-intensive, and difficult to scale. Although recent advances in Large…

Software Engineering · Computer Science 2025-12-17 Ruozhao Yang , Mingfei Cheng , Gelei Deng , Tianwei Zhang , Junjie Wang , Xiaofei Xie

We present MEGA-Bench, an evaluation suite that scales multimodal evaluation to over 500 real-world tasks, to address the highly heterogeneous daily use cases of end users. Our objective is to optimize for a set of high-quality data samples…

Computer Vision and Pattern Recognition · Computer Science 2025-07-15 Jiacheng Chen , Tianhao Liang , Sherman Siu , Zhengqing Wang , Kai Wang , Yubo Wang , Yuansheng Ni , Wang Zhu , Ziyan Jiang , Bohan Lyu , Dongfu Jiang , Xuan He , Yuan Liu , Hexiang Hu , Xiang Yue , Wenhu Chen

Real-world autonomous planning requires coordinating tightly coupled constraints where a single decision dictates the feasibility of all subsequent actions. However, existing benchmarks predominantly feature loosely coupled constraints…

With the deep integration of artificial intelligence and interactive technology, Graphical User Interface (GUI) Agent, as the carrier connecting goal-oriented natural language and real-world devices, has received widespread attention from…

Artificial Intelligence · Computer Science 2025-11-13 Leyang Yang , Ziwei Wang , Xiaoxuan Tang , Sheng Zhou , Dajun Chen , Wei Jiang , Yong Li

We present a benchmark targeting a novel class of systems: semantic query processing engines. Those systems rely inherently on generative and reasoning capabilities of state-of-the-art large language models (LLMs). They extend SQL with…

The evolution of Large Language Models (LLMs) into autonomous agents has expanded the scope of AI coding from localized code generation to complex, repository-level, and execution-driven problem solving. However, current benchmarks…

Software Engineering · Computer Science 2026-01-19 Jie Yang , Honglin Guo , Li Ji , Jiazheng Zhou , Rui Zheng , Zhikai Lei , Shuo Zhang , Zhiheng Xi , Shichun Liu , Yuxin Wang , Bo Wang , Yining Zheng , Tao Gui , Xipeng Qiu

Significant focus has been placed on integrating large language models (LLMs) with various tools in developing general-purpose agents. This poses a challenge to LLMs' tool-use capabilities. However, there are evident gaps between existing…

Computation and Language · Computer Science 2024-11-25 Jize Wang , Zerun Ma , Yining Li , Songyang Zhang , Cailian Chen , Kai Chen , Xinyi Le

Decompilers are fundamental tools for critical security tasks, from vulnerability discovery to malware analysis, yet their evaluation remains fragmented. Existing approaches primarily focus on syntactic correctness through synthetic…

Software Engineering · Computer Science 2025-05-19 Zeyu Gao , Yuxin Cui , Hao Wang , Siliang Qin , Yuanda Wang , Bolun Zhang , Chao Zhang