English
Related papers

Related papers: A3: Android Agent Arena for Mobile GUI Agents with…

200 papers

Mobile GUI agents powered by large language models have progressed rapidly, creating urgent needs for realistic and comprehensive evaluation. Existing benchmarks prioritize reproducibility but are often limited to open-source apps or…

Artificial Intelligence · Computer Science 2026-05-26 Guohong Liu , Jialei Ye , Pengzhi Gao , Wei Liu , Jian Luan , Yunxin Liu , Yuanchun Li

Despite rapid development, large language models (LLMs) still encounter challenges in multi-turn decision-making tasks (i.e., agent tasks) like web shopping and browser navigation, which require making a sequence of intelligent decisions…

Computation and Language · Computer Science 2025-11-12 Zhiheng Xi , Chenyang Liao , Guanyu Li , Yajie Yang , Wenxiang Chen , Zhihao Zhang , Binghai Wang , Senjie Jin , Yuhao Zhou , Jian Guan , Wei Wu , Tao Ji , Tao Gui , Qi Zhang , Xuanjing Huang

Autonomous agents that navigate Graphical User Interfaces (GUIs) to automate tasks like document editing and file management can greatly enhance computer workflows. While existing research focuses on online settings, desktop environments,…

Mobile graphical user interface (GUI) agents enable AI models to autonomously operate smartphones on behalf of users. However, most existing systems focus primarily on optimizing task accuracy and rely on cloud-hosted models for inference,…

Artificial Intelligence · Computer Science 2026-05-27 Runxi Huang , Liyu Zhang , Shengzhong Liu , Xiaomin Ouyang

Understanding and reasoning on the large-scale scientific literature is a crucial touchstone for large language model (LLM) based agents. However, existing works are mainly restricted to tool-free tasks within single papers, largely due to…

Artificial Intelligence · Computer Science 2026-02-02 Daoyu Wang , Mingyue Cheng , Shuo Yu , Zirui Liu , Ze Guo , Xin Li , Qi Liu

Graphical user interface (GUI) agents are rapidly progressing toward autonomous interaction and reliable task execution across diverse applications. However, two central challenges remain unresolved: automating the evaluation of agent…

The emergence of multimodal LLM-based agents (MLAs) has transformed interaction paradigms by seamlessly integrating vision, language, action and dynamic environments, enabling unprecedented autonomous capabilities across GUI applications…

Artificial Intelligence · Computer Science 2025-06-03 Xiao Yang , Jiawei Chen , Jun Luo , Zhengwei Fang , Yinpeng Dong , Hang Su , Jun Zhu

Mobile device control agents can largely enhance user interactions and productivity by automating daily tasks. However, despite growing interest in developing practical agents, the absence of a commonly adopted benchmark in this area makes…

Human-Computer Interaction · Computer Science 2025-07-22 Juyong Lee , Taywon Min , Minyong An , Dongyoon Hahm , Haeone Lee , Changyeon Kim , Kimin Lee

Autonomous Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) enable digital automation on end-user devices. While scaling both parameters and data has yielded substantial gains, advanced methods still…

Artificial Intelligence · Computer Science 2026-04-16 Ziwei Wang , Junjie Zheng , Leyang Yang , Sheng Zhou , Xiaoxuan Tang , Zhouhua Fang , Zhiwei Liu , Dajun Chen , Yong Li , Jiajun Bu

Large Language Models (LLMs) are transforming artificial intelligence, enabling autonomous agents to perform diverse tasks across various domains. These agents, proficient in human-like text comprehension and generation, have the potential…

Artificial Intelligence · Computer Science 2024-04-10 Saikat Barua

As LLMs are increasingly deployed as agents, agentic reasoning - the ability to combine tool use, especially search, and reasoning - becomes a critical skill. However, it is hard to disentangle agentic reasoning when evaluated in complex…

Artificial Intelligence · Computer Science 2025-10-03 Hanlin Zhu , Tianyu Guo , Song Mei , Stuart Russell , Nikhil Ghosh , Alberto Bietti , Jiantao Jiao

Game-theoretic scenarios have become pivotal in evaluating the social intelligence of Large Language Model (LLM)-based social agents. While numerous studies have explored these agents in such settings, there is a lack of a comprehensive…

Computation and Language · Computer Science 2025-07-22 Xiachong Feng , Longxu Dou , Ella Li , Qinghao Wang , Haochuan Wang , Yu Guo , Chang Ma , Lingpeng Kong

A/B testing experiment is a widely adopted method for evaluating UI/UX design decisions in modern web applications. Yet, traditional A/B testing remains constrained by its dependence on the large-scale and live traffic of human…

Human-Computer Interaction · Computer Science 2026-03-12 Yuxuan Lu , Ting-Yao Hsu , Hansu Gu , Limeng Cui , Yaochen Xie , William Headden , Bingsheng Yao , Akash Veeragouni , Jiapeng Liu , Sreyashi Nag , Jessie Wang , Dakuo Wang

The advent of large language models (LLMs), such as GPT, Gemini, and DeepSeek, has significantly advanced natural language processing, giving rise to sophisticated chatbots capable of diverse language-related tasks. The transition from…

Computation and Language · Computer Science 2025-06-16 Jiachen Zhu , Menghui Zhu , Renting Rui , Rong Shan , Congmin Zheng , Bo Chen , Yunjia Xi , Jianghao Lin , Weiwen Liu , Ruiming Tang , Yong Yu , Weinan Zhang

We present TextAtari, a benchmark for evaluating language agents on very long-horizon decision-making tasks spanning up to 100,000 steps. By translating the visual state representations of classic Atari games into rich textual descriptions,…

Computation and Language · Computer Science 2025-06-11 Wenhao Li , Wenwu Li , Chuyun Shen , Junjie Sheng , Zixiao Huang , Di Wu , Yun Hua , Wei Yin , Xiangfeng Wang , Hongyuan Zha , Bo Jin

Large Action Models (LAMs) for AI Agents offer incredible potential but face challenges due to the need for high-quality training data, especially for multi-steps tasks that involve planning, executing tool calls, and responding to…

The potential of Large Language Model (LLM) as agents has been widely acknowledged recently. Thus, there is an urgent need to quantitatively \textit{evaluate LLMs as agents} on challenging tasks in interactive environments. We present…

The development of LLM-based autonomous agents for end-to-end software development represents a significant paradigm shift in software engineering. However, the scientific evaluation of these systems is hampered by significant challenges,…

Software Engineering · Computer Science 2025-11-07 Zhengran Zeng , Yixin Li , Rui Xie , Wei Ye , Shikun Zhang

Agentic systems are becoming more capable: agents define strategies, take actions, and interact with different environments. This autonomy poses serious challenges for overseeing and assessing agent behavior. Most current tools are limited,…

Computation and Language · Computer Science 2026-05-22 Asaf Yehudai , Lilach Eden , Michal Shmueli-Scheuer

Mobile Graphical User Interface (GUI) agents powered by multimodal large language models have demonstrated promising capabilities in automating complex smartphone tasks. However, existing approaches face two critical limitations: the…

Computer Vision and Pattern Recognition · Computer Science 2026-04-01 Yiping Xie , Song Chen , Jingxuan Xing , Wei Jiang , Zekun Zhu , Yingyao Wang , Pi Bu , Jun Song , Yuning Jiang , Bo Zheng