English
Related papers

Related papers: How Brittle is Agent Safety? Rethinking Agent Risk…

200 papers

We take the position that agent security must be approached as a systems problem: the AI model powering the agent must be treated as an untrusted component, and security invariants must be enforced at the system level. Through this lens,…

Current test-time scaling (TTS) techniques enhance large language model (LLM) performance by allocating additional computation at inference time, yet they remain insufficient for agentic settings, where actions directly interact with…

Computation and Language · Computer Science 2026-02-04 Xingshan Zeng , Lingzhi Wang , Weiwen Liu , Liangyou Li , Yasheng Wang , Lifeng Shang , Xin Jiang , Qun Liu

LLM agents have begun to appear as personal assistants, customer service bots, and clinical aides. While these applications deliver substantial operational benefits, they also require continuous access to sensitive data, which increases the…

Cryptography and Security · Computer Science 2025-09-30 Saswat Das , Jameson Sandler , Ferdinando Fioretto

Security Operations Centers (SOCs) are overwhelmed by tens of thousands of daily alerts, with only a small fraction corresponding to genuine attacks. This overload creates alert fatigue, leading to overlooked threats and analyst burnout.…

Computation and Language · Computer Science 2025-10-02 Bowen Wei , Yuan Shen Tay , Howard Liu , Jinhao Pan , Kun Luo , Ziwei Zhu , Chris Jordan

Recent advancements in Web AI agents have demonstrated remarkable capabilities in addressing complex web navigation tasks. However, emerging research shows that these agents exhibit greater vulnerability compared to standalone Large…

Machine Learning · Computer Science 2025-09-23 Jeffrey Yang Fan Chiang , Seungjae Lee , Jia-Bin Huang , Furong Huang , Yizheng Chen

Autonomous Large Language Model (LLM) agents, exemplified by OpenClaw, demonstrate remarkable capabilities in executing complex, long-horizon tasks. However, their tightly coupled instant-messaging interaction paradigm and high-privilege…

The rapid rise of autonomous AI systems and advancements in agent capabilities are introducing new risks due to reduced oversight of real-world interactions. Yet agent testing remains nascent and is still a developing science. As AI agents…

Addressing contextual privacy concerns remains challenging in interactive settings where large language models (LLMs) process information from multiple sources (e.g., summarizing meetings with private and public information). We introduce a…

Artificial Intelligence · Computer Science 2026-02-26 Wenkai Li , Liwen Sun , Zhenxiang Guan , Xuhui Zhou , Maarten Sap

Security in LLM agents is inherently contextual. For example, the same action taken by an agent may represent legitimate behavior or a security violation depending on whose instruction led to the action, what objective is being pursued, and…

Cryptography and Security · Computer Science 2026-03-23 Vincent Siu , Jingxuan He , Kyle Montgomery , Zhun Wang , Neil Gong , Chenguang Wang , Dawn Song

Large Language Model agents deployed in complex environments frequently encounter a conflict between maximizing goal achievement and adhering to safety constraints. This paper identifies a new concept called Agentic Pressure, which…

Artificial Intelligence · Computer Science 2026-04-21 Hengle Jiang , Ke Tang

Open agentic systems combine LLM-based planning with external capabilities, persistent memory, and privileged execution. They are used in coding assistants, browser copilots, and enterprise automation. OpenClaw is a visible instance of this…

Cryptography and Security · Computer Science 2026-03-30 Shiping Chen , Qin Wang , Guangsheng Yu , Xu Wang , Liming Zhu

AI agents are increasingly deployed in multi-task settings, where the task to perform is specified at test time, and the agent must generalize to unseen tasks. A major concern in such settings is safety: often, an agent must not only…

Machine Learning · Computer Science 2026-05-11 Yonatan Slutzky , Yotam Alexander , Tomer Slor , Yoav Nagel , Nadav Cohen

Autonomous AI agents powered by Large Language Models can reason, plan, and execute complex tasks, but their ability to autonomously retrieve information and run code introduces significant security risks. Existing approaches attempt to…

Cryptography and Security · Computer Science 2026-04-09 Hongyi Lu , Nian Liu , Shuai Wang , Fengwei Zhang

As large language models (LLMs) are increasingly deployed as agents, their integration into interactive environments and tool use introduce new safety challenges beyond those associated with the models themselves. However, the absence of…

Computation and Language · Computer Science 2025-05-21 Zhexin Zhang , Shiyao Cui , Yida Lu , Jingzhuo Zhou , Junxiao Yang , Hongning Wang , Minlie Huang

Indirect prompt injection attacks threaten AI agents that execute consequential actions, motivating deterministic system-level defenses. Such defenses can provably block unsafe actions by enforcing confidentiality and integrity policies,…

Cryptography and Security · Computer Science 2026-02-13 Aashish Kolluri , Rishi Sharma , Manuel Costa , Boris Köpf , Tobias Nießen , Mark Russinovich , Shruti Tople , Santiago Zanella-Béguelin

Indirect prompt injection threatens LLM agents by embedding malicious instructions in external content, enabling unauthorized actions and data theft. LLM agents maintain working memory through their context window, which stores interaction…

Cryptography and Security · Computer Science 2026-02-10 Ruoyao Wen , Hao Li , Chaowei Xiao , Ning Zhang

We argue that LLM agent security is fundamentally an agent-human interaction (AHI) problem, not a purely algorithmic one. To substantiate this position, we conduct a systematic analysis of 59 academic papers, 21 production agent systems,…

Cryptography and Security · Computer Science 2026-05-26 Peiran Wang , Ying Li , Yuan Tian

Large Language Models (LLMs) have demonstrated strong capabilities as autonomous agents through tool use, planning, and decision-making abilities, leading to their widespread adoption across diverse tasks. As task complexity grows,…

Multiagent Systems · Computer Science 2025-11-10 Ishan Kavathekar , Hemang Jain , Ameya Rathod , Ponnurangam Kumaraguru , Tanuja Ganu

The increasing adoption of Reinforcement Learning in safety-critical systems domains such as autonomous vehicles, health, and aviation raises the need for ensuring their safety. Existing safety mechanisms such as adversarial training,…

Machine Learning · Computer Science 2021-11-11 Paulina Stevia Nouwou Mindom , Amin Nikanjam , Foutse Khomh , John Mullins

Personal AI agents like OpenClaw run with elevated privileges on users' local machines, where a single successful prompt injection can leak credentials, redirect financial transactions, or destroy files. This threat goes well beyond…

Artificial Intelligence · Computer Science 2026-04-07 Bowen Wei , Yunbei Zhang , Jinhao Pan , Kai Mei , Xiao Wang , Jihun Hamm , Ziwei Zhu , Yingqiang Ge