English
Related papers

Related papers: AgentHijack: Benchmarking Computer Use Agent Robus…

200 papers

Recent years have witnessed a rapid development of mobile GUI agents powered by large language models (LLMs), which can autonomously execute diverse device-control tasks based on natural language instructions. The increasing accuracy of…

Cryptography and Security · Computer Science 2026-04-15 Guohong Liu , Jialei Ye , Jiacheng Liu , Yuanchun Li , Wei Liu , Pengzhi Gao , Jian Luan , Yunxin Liu

Web agents enable users to perform tasks on web browsers through natural language interaction. Evaluating web agents trajectories is an important problem, since it helps us determine whether the agent successfully completed the tasks.…

Large language model (LLM)-based mobile agents are increasingly popular due to their capability to interact directly with mobile phone Graphic User Interfaces (GUIs) and their potential to autonomously manage daily tasks. Despite their…

Artificial Intelligence · Computer Science 2024-06-13 Luyuan Wang , Yongyu Deng , Yiwei Zha , Guodong Mao , Qinmin Wang , Tianchen Min , Wei Chen , Shoufa Chen

Multi-agent systems coordinate LLM-based agents to perform tasks on users' behalf. In real-world applications, multi-agent systems will inevitably interact with untrusted inputs, such as malicious Web content, files, email attachments, and…

Cryptography and Security · Computer Science 2025-09-16 Harold Triedman , Rishi Jha , Vitaly Shmatikov

With the rapid development of multimodal large language models (MLLMs), they are increasingly deployed as autonomous computer-use agents capable of accomplishing complex computer tasks. However, a pressing issue arises: Can the safety risk…

Artificial Intelligence · Computer Science 2025-06-23 Jingyi Yang , Shuai Shao , Dongrui Liu , Jing Shao

Large language models are increasingly proposed as autonomous agents for high-stakes public workflows, yet we lack systematic evidence about whether they would follow institutional rules when granted authority. We present evidence that…

Artificial Intelligence · Computer Science 2026-03-20 Vedanta S P , Ponnurangam Kumaraguru

The advances made by Large Language Models (LLMs) have led to the pursuit of LLM agents that can solve intricate, multi-step reasoning tasks. As with any research pursuit, benchmarking and evaluation are key corner stones to efficient and…

Artificial Intelligence · Computer Science 2024-04-10 Luca Gioacchini , Giuseppe Siracusano , Davide Sanvito , Kiril Gashteovski , David Friede , Roberto Bifulco , Carolin Lawrence

Agent benchmarks have become the de facto measure of frontier AI competence, guiding model selection, investment, and deployment. However, reward hacking, where agents maximize a score without performing the intended task, emerges…

Artificial Intelligence · Computer Science 2026-05-14 Hao Wang , Hanchen Li , Qiuyang Mang , Alvin Cheung , Koushik Sen , Dawn Song

As large language models (LLMs) are increasingly deployed as agents, their integration into interactive environments and tool use introduce new safety challenges beyond those associated with the models themselves. However, the absence of…

Computation and Language · Computer Science 2025-05-21 Zhexin Zhang , Shiyao Cui , Yida Lu , Jingzhuo Zhou , Junxiao Yang , Hongning Wang , Minlie Huang

LLM agents are increasingly deployed in long-horizon, complex environments to solve challenging problems, but this expansion exposes them to long-horizon attacks that exploit multi-turn user-agent-environment interactions to achieve…

Artificial Intelligence · Computer Science 2026-02-20 Tanqiu Jiang , Yuhui Wang , Jiacheng Liang , Ting Wang

Among existing online mobile-use benchmarks, AndroidWorld has emerged as the dominant benchmark due to its reproducible environment and deterministic evaluation; however, recent agents achieving over 90% success rates indicate its…

Computation and Language · Computer Science 2026-01-01 Quyu Kong , Xu Zhang , Zhenyu Yang , Nolan Gao , Chen Liu , Panrong Tong , Chenglin Cai , Hanzhang Zhou , Jianan Zhang , Liangyu Chen , Zhidan Liu , Steven Hoi , Yue Wang

We present HippoCamp, a new benchmark designed to evaluate agents' capabilities on multimodal file management. Unlike existing agent benchmarks that focus on tasks like web interaction, tool use, or software automation in generic settings,…

Artificial Intelligence · Computer Science 2026-04-02 Zhe Yang , Shulin Tian , Kairui Hu , Shuai Liu , Hoang-Nhat Nguyen , Yichi Zhang , Zujin Guo , Mengying Yu , Zinan Zhang , Jingkang Yang , Chen Change Loy , Ziwei Liu

LLM agents increasingly perform end-to-end ML engineering tasks where success is judged by a single scalar test metric. This creates a structural vulnerability: an agent can increase the reported score by compromising the evaluation…

Artificial Intelligence · Computer Science 2026-03-13 Yonas Atinafu , Robin Cohen

Large Language Models (LLMs) have demonstrated strong capabilities as autonomous agents through tool use, planning, and decision-making abilities, leading to their widespread adoption across diverse tasks. As task complexity grows,…

Multiagent Systems · Computer Science 2025-11-10 Ishan Kavathekar , Hemang Jain , Ameya Rathod , Ponnurangam Kumaraguru , Tanuja Ganu

Computer-use agent (CUA) frameworks, powered by large language models (LLMs) or multimodal LLMs (MLLMs), are rapidly maturing as assistants that can perceive context, reason, and act directly within software environments. Among their most…

Cryptography and Security · Computer Science 2025-10-13 Weidi Luo , Qiming Zhang , Tianyu Lu , Xiaogeng Liu , Bin Hu , Hung-Chun Chiu , Siyuan Ma , Yizhe Zhang , Xusheng Xiao , Yinzhi Cao , Zhen Xiang , Chaowei Xiao

Large Language Models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have demonstrated strong reasoning capabilities. To further enhance LLM capabilities, recent agentic systems, such as Deep Research, incorporate web interactions into LLM…

Artificial Intelligence · Computer Science 2025-10-21 Song Bian , Minghao Yan , Anand Jayarajan , Gennady Pekhimenko , Shivaram Venkataraman

Third-party skills are becoming the package ecosystem for LLM agents. They package natural-language instructions, helper scripts, templates, documents, and service configuration into reusable workflows. This makes skills useful, but it also…

Cryptography and Security · Computer Science 2026-05-15 Haomin Zhuang , Hanwen Xing , Yujun Zhou , Yuchen Ma , Yue Huang , Yili Shen , Yufei Han , Xiangliang Zhang

Recent advances in AI agents capable of solving complex, everyday tasks, from scheduling to customer service, have enabled deployment in real-world settings, but their possibilities for unsafe behavior demands rigorous evaluation. While…

Artificial Intelligence · Computer Science 2026-02-18 Sanidhya Vijayvargiya , Aditya Bharat Soni , Xuhui Zhou , Zora Zhiruo Wang , Nouha Dziri , Graham Neubig , Maarten Sap

Large Language Models (LLMs) are becoming increasingly powerful and capable of handling complex tasks, e.g., building single agents and multi-agent systems. Compared to single agents, multi-agent systems have higher requirements for the…

Computation and Language · Computer Science 2024-08-29 Wei Wang , Dan Zhang , Tao Feng , Boyan Wang , Jie Tang

Agentic AI systems -- Large Language Models (LLMs) augmented with planning, tool use, memory, and long-horizon interactions -- can execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that…

Artificial Intelligence · Computer Science 2026-05-26 Jinhu Qi , Muzhi Li , Jiahong Liu , Yuqin Shu , Dianzhi Yu , Shicheng Ma , Wenqian Cui , Yiyang Zhao , Yiyi Chen , Ruoxi Jiang , Irwin King , Zenglin Xu