English
Related papers

Related papers: Owner-Harm: A Missing Threat Model for AI Agent Sa…

200 papers

Safety risks of AI models have been widely studied at deployment time, such as jailbreak attacks that elicit harmful outputs. In contrast, safety risks emerging during training remain largely unexplored. Beyond explicit reward hacking that…

Computation and Language · Computer Science 2026-02-05 Zhexin Zhang , Yida Lu , Junfeng Fang , Junxiao Yang , Shiyao Cui , Hao Zhou , Fandong Meng , Jie Zhou , Hongning Wang , Minlie Huang , Tat-Seng Chua

We analyzed 3,984 AI agent skills from major marketplaces and found 76 confirmed malicious payloads, including credential theft, backdoor installation, and data exfiltration. 13.4% of all skills contain at least one critical-level security…

Cryptography and Security · Computer Science 2026-05-28 Luca Beurer-Kellner , Aleksei Kudrinskii , Marco Milanta , Kristian Bonde Nielsen , Hemang Sarkar , Liran Tal

Agentic AI systems -- Large Language Models (LLMs) augmented with planning, tool use, memory, and long-horizon interactions -- can execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that…

Artificial Intelligence · Computer Science 2026-05-26 Jinhu Qi , Muzhi Li , Jiahong Liu , Yuqin Shu , Dianzhi Yu , Shicheng Ma , Wenqian Cui , Yiyang Zhao , Yiyi Chen , Ruoxi Jiang , Irwin King , Zenglin Xu

With the widespread application of Large Language Models (LLMs), their associated security issues have become increasingly prominent, severely constraining their trustworthy deployment in critical domains. This paper proposes a novel safety…

Artificial Intelligence · Computer Science 2025-11-18 Qi Li , Jianjun Xu , Pingtao Wei , Jiu Li , Peiqiang Zhao , Jiwei Shi , Xuan Zhang , Yanhui Yang , Xiaodong Hui , Peng Xu , Wenqin Shao

As autonomous agents (e.g., OpenClaw) increasingly operate with deep system-level privileges to execute complex tasks, they introduce severe, unmitigated security risks. Current vulnerability analyses overwhelmingly focus on single-turn,…

Cryptography and Security · Computer Science 2026-05-22 Jianan Ma , Xiaohu Du , Ruixiao Lin , Yaoxiang Bian , Jialuo Chen , Jingyi Wang , Xiaofang Yang , Shiwen Cui , Changhua Meng , Xinhao Deng , Zhen Wang

As the industry increasingly adopts agentic AI systems, understanding their unique vulnerabilities becomes critical. Prior research suggests that security flaws at the model level do not fully capture the risks present in agentic…

Artificial Intelligence · Computer Science 2025-09-23 Ilham Wicaksono , Zekun Wu , Rahul Patel , Theo King , Adriano Koshiyama , Philip Treleaven

Previous benchmarks on prompt injection in large language models (LLMs) have primarily focused on generic tasks and attacks, offering limited insights into more complex threats like data exfiltration. This paper examines how prompt…

Cryptography and Security · Computer Science 2025-06-03 Meysam Alizadeh , Zeynab Samei , Daria Stetsenko , Fabrizio Gilardi

As AI systems grow more capable and autonomous, ensuring their safety and reliability requires not only model-level alignment but also strategic oversight of the humans and institutions involved in their development and deployment. Existing…

Artificial Intelligence · Computer Science 2026-02-10 Cheol Woo Kim , Davin Choo , Tzeh Yuan Neoh , Milind Tambe

Agentic AI systems, specifically LLM-driven agents that plan, invoke tools, maintain persistent memory, and delegate tasks to peer agents via protocols such as MCP and A2A, introduce a threat surface that differs materially from standalone…

Cryptography and Security · Computer Science 2026-05-08 Javad Forough , Marios Kogias , Hamed Haddadi

The integration of artificial intelligence (AI) agents into web browsers introduces security challenges that go beyond traditional web application threat models. Prior work has identified prompt injection as a new attack vector for web…

Machine Learning · Computer Science 2025-11-26 Kaiyuan Zhang , Mark Tenenholtz , Kyle Polley , Jerry Ma , Denis Yarats , Ninghui Li

LLM agents are evolving rapidly, powered by code execution, tools, and the recently introduced agent skills feature. Skills allow users to extend LLM applications with specialized third-party code, knowledge, and instructions. Although this…

Cryptography and Security · Computer Science 2026-02-26 David Schmotz , Luca Beurer-Kellner , Sahar Abdelnabi , Maksym Andriushchenko

AI agents are vulnerable to indirect prompt injection attacks, where malicious instructions embedded in external content or tool outputs cause unintended or harmful behavior. Inspired by the well-established concept of firewalls, we show…

Cryptography and Security · Computer Science 2026-03-24 Rishika Bhagwatkar , Kevin Kasa , Abhay Puri , Gabriel Huang , Irina Rish , Graham W. Taylor , Krishnamurthy Dj Dvijotham , Alexandre Lacoste

Large language model-based agents are rapidly evolving from simple conversational assistants into autonomous systems capable of performing complex, professional-level tasks in various domains. While these advancements promise significant…

The rapid advancement of large models, driven by their exceptional abilities in learning and generalization through large-scale pre-training, has reshaped the landscape of Artificial Intelligence (AI). These models are now foundational to a…

AI agents are beginning to interact with each other directly and across internet platforms and physical environments, creating security challenges beyond traditional cybersecurity and AI safety frameworks. Free-form protocols are essential…

Backdoor attacks pose a serious threat to the secure deployment of large language models (LLMs), enabling adversaries to implant hidden behaviors triggered by specific inputs. However, existing methods often rely on manually crafted…

Cryptography and Security · Computer Science 2025-11-24 Yige Li , Zhe Li , Wei Zhao , Nay Myat Min , Hanxun Huang , Xingjun Ma , Jun Sun

Autonomous UI agents powered by AI have tremendous potential to boost human productivity by automating routine tasks such as filing taxes and paying bills. However, a major challenge in unlocking their full potential is security, which is…

Cryptography and Security · Computer Science 2025-05-20 Ivan Evtimov , Arman Zharmagambetov , Aaron Grattafiori , Chuan Guo , Kamalika Chaudhuri

Autonomous AI agents increasingly extend their capabilities through Agent Skills: modular filesystem packages whose SKILL.md files describe when and how agents should use them. While this design enables scalable, on-demand capability…

Artificial Intelligence · Computer Science 2026-05-13 Shoumik Saha , Kazem Faghih , Soheil Feizi

While finetuning AI agents on interaction data -- such as web browsing or tool use -- improves their capabilities, it also introduces critical security vulnerabilities within the agentic AI supply chain. We show that adversaries can…

Agentic AI systems increasingly act through tool-augmented, multi-step workflows whose failures (unsafe tool use, unauthorised actions, social harm) carry deployment-level consequences. Evaluation practice remains fragmented across isolated…

Computation and Language · Computer Science 2026-05-22 Jinhu Qi , Yifan Li , Minghao Zhao , Wentao Zhang , Zijian Zhang , Yaoman Li , Irwin King