English
Related papers

Related papers: Red-Teaming Agent Execution Contexts: Open-World S…

200 papers

Language model (LM) agents have demonstrated significant potential for automating real-world tasks, yet they pose a diverse array of potential, severe risks in safety-critical scenarios. In this work, we identify a significant gap between…

Artificial Intelligence · Computer Science 2025-08-20 Yuzhi Tang , Tianxiao Li , Elizabeth Li , Chris J. Maddison , Honghua Dong , Yangjun Ruan

Large language model (LLM) agents are vulnerable to prompt-injection attacks that propagate through multi-step workflows, tool interactions, and persistent context, making input-output filtering alone insufficient for reliable protection.…

Artificial Intelligence · Computer Science 2026-04-21 Hailin Liu , Eugene Ilyushin , Jie Ni , Min Zhu

Agentic systems are becoming more capable: agents define strategies, take actions, and interact with different environments. This autonomy poses serious challenges for overseeing and assessing agent behavior. Most current tools are limited,…

Computation and Language · Computer Science 2026-05-22 Asaf Yehudai , Lilach Eden , Michal Shmueli-Scheuer

As multi-agent AI systems are increasingly deployed in real-world settings - from automated customer support to DevOps remediation - failures become harder to diagnose due to cascading effects, hidden dependencies, and long execution…

Machine Learning · Computer Science 2026-03-30 Zhaohui Geoffrey Wang

Personalized computer-use agents are rapidly moving from expert communities into mainstream use. Unlike conventional chatbots, these systems can install skills, invoke tools, access private resources, and modify local environments on users'…

Cryptography and Security · Computer Science 2026-04-01 Zifan Peng , Mingchen Li

Agents built on LLMs are increasingly deployed across diverse domains, automating complex decision-making and task execution. However, their autonomy introduces safety risks, including security vulnerabilities, legal violations, and…

Artificial Intelligence · Computer Science 2025-08-01 Haoyu Wang , Christopher M. Poskitt , Jun Sun

Agentic AI systems automate enterprise workflows but existing defenses--guardrails, semantic filters--are probabilistic and routinely bypassed. We introduce authenticated workflows, the first complete trust layer for enterprise agentic AI.…

Cryptography and Security · Computer Science 2026-02-12 Mohan Rajagopalan , Vinay Rao

The rapid deployment of Large language model (LLM) agents in critical domains like healthcare and finance necessitates robust security frameworks. To address the absence of standardized evaluation benchmarks for these agents in dynamic…

Cryptography and Security · Computer Science 2025-06-19 Yuchuan Fu , Xiaohan Yuan , Dongxia Wang

Task success can hide process anomalies in real-world agent executions. An agent may pass the final task oracle while still accumulating unresolved ambiguity, unsafe external writes, ignored errors, weakly grounded commitments, or…

Artificial Intelligence · Computer Science 2026-05-29 Yibing Liu , Yangze Liu , Xiaolong Yin , Bin Wang , Chong Zhang , Hao Yin , Zhongyi Han

Adversarial examples can be useful for identifying vulnerabilities in AI systems before they are deployed. In reinforcement learning (RL), adversarial policies can be developed by training an adversarial agent to minimize a target agent's…

Artificial Intelligence · Computer Science 2023-10-17 Stephen Casper , Taylor Killian , Gabriel Kreiman , Dylan Hadfield-Menell

Agentic AI systems -- Large Language Models (LLMs) augmented with planning, tool use, memory, and long-horizon interactions -- can execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that…

Artificial Intelligence · Computer Science 2026-05-26 Jinhu Qi , Muzhi Li , Jiahong Liu , Yuqin Shu , Dianzhi Yu , Shicheng Ma , Wenqian Cui , Yiyang Zhao , Yiyi Chen , Ruoxi Jiang , Irwin King , Zenglin Xu

Backdoor attacks pose a serious threat to the secure deployment of large language models (LLMs), enabling adversaries to implant hidden behaviors triggered by specific inputs. However, existing methods often rely on manually crafted…

Cryptography and Security · Computer Science 2025-11-24 Yige Li , Zhe Li , Wei Zhao , Nay Myat Min , Hanxun Huang , Xingjun Ma , Jun Sun

LLM-based agent systems increasingly rely on agent skills sourced from open registries to extend their capabilities, yet the openness of such ecosystems makes skills difficult to thoroughly vet. Existing attacks rely on injecting malicious…

Cryptography and Security · Computer Science 2026-04-08 Zenghao Duan , Yuxin Tian , Zhiyi Yin , Liang Pang , Jingcheng Deng , Zihao Wei , Shicheng Xu , Yuyao Ge , Xueqi Cheng

Large language models are increasingly deployed as *deep agents* that plan, maintain persistent state, and invoke external tools, shifting safety failures from unsafe text to unsafe *trajectories*. We introduce **AgentFence**, an…

Cryptography and Security · Computer Science 2026-02-10 Sai Puppala , Ismail Hossain , Md Jahangir Alam , Yoonpyo Lee , Jay Yoo , Tanzim Ahad , Syed Bahauddin Alam , Sajedul Talukder

As Large Language Models (LLMs) become increasingly integrated into real-world decision-making systems, understanding their behavioural vulnerabilities remains a critical challenge for AI safety and alignment. While existing evaluation…

Artificial Intelligence · Computer Science 2025-05-20 Lili Zhang , Haomiaomiao Wang , Long Cheng , Libao Deng , Tomas Ward

The rise of Large Language Model (LLM) agents, augmented with tool use, skills, and external knowledge, has introduced new security risks. Among them, prompt injection attacks, where adversaries embed malicious instructions into the agent…

Cryptography and Security · Computer Science 2026-05-06 Shihao Weng , Yang Feng , Jinrui Zhang , Xiaofei Xie , Jiongchi Yu , Jia Liu

Language agents have shown impressive problem-solving skills within defined settings and brief timelines. Yet, with the ever-evolving complexities of open-world simulations, there's a pressing need for agents that can flexibly adapt to…

Artificial Intelligence · Computer Science 2024-01-01 Ming Yan , Ruihao Li , Hao Zhang , Hao Wang , Zhilan Yang , Ji Yan

OpenClaw-like agents offer substantial productivity benefits, yet they are insecure by default because they combine untrusted inputs, autonomous action, extensibility, and privileged system access within a single execution loop. We use…

Cryptography and Security · Computer Science 2026-03-16 Zongwei Li , Wenkai Li , Xiaoqi Li

We propose an extension to the OWASP Multi-Agentic System (MAS) Threat Modeling Guide, translating recent anticipatory research in multi-agent security (MASEC) into practical guidance for addressing challenges unique to large language model…

Multiagent Systems · Computer Science 2025-08-14 Klaudia Krawiecka , Christian Schroeder de Witt

As large language models (LLMs) improve, so do their offensive applications: frontier agents now generate working exploits for under $50 in compute (Heelan, 2026). Defensive incident response (IR) agents must keep pace, but existing…

Artificial Intelligence · Computer Science 2026-02-10 Jarrod Barnes