English
Related papers

Related papers: How Brittle is Agent Safety? Rethinking Agent Risk…

200 papers

Large language model (LLM) agents with extended autonomy unlock new capabilities, but also introduce heightened challenges for LLM safety. In particular, an LLM agent may pursue objectives that deviate from human values and ethical norms, a…

Computation and Language · Computer Science 2026-01-27 Chen Chen , Kim Young Il , Yuan Yang , Wenhao Su , Yilin Zhang , Xueluan Gong , Qian Wang , Yongsen Zheng , Ziyao Liu , Kwok-Yan Lam

The rapid deployment of open-source frameworks has significantly advanced the development of modern multi-agent systems. However, expanded action spaces, including uncontrolled privilege exposure and hidden inter-system interactions, pose…

Computation and Language · Computer Science 2026-04-07 Wenhui Zhu , Xuanzhao Dong , Xiwen Chen , Rui Cai , Peijie Qiu , Zhipeng Wang , Oana Frunza , Shao Tang , Jindong Gu , Yalin Wang

The popularity of large language models (LLMs) continues to grow, and LLM-based assistants have become ubiquitous. Information security awareness (ISA) is an important yet underexplored area of LLM safety. ISA encompasses LLMs' security…

Cryptography and Security · Computer Science 2026-03-23 Ofir Cohen , Gil Ari Agmon , Asaf Shabtai , Rami Puzis

Large language model (LLM)-based agents combine LLMs with external tools to automate tasks such as scheduling meetings, managing documents, or booking travel. While these integrations unlock powerful capabilities, they also create new and…

Cryptography and Security · Computer Science 2026-04-22 Jonathan Evertz , Merlin Chlosta , Lea Schönherr , Thorsten Eisenhofer

Large Language Model (LLM) agents can leverage tools such as Google Search to complete complex tasks. However, this tool usage introduces the risk of indirect prompt injections, where malicious instructions hidden in tool outputs can…

Machine Learning · Computer Science 2025-10-08 Zizhao Wang , Dingcheng Li , Vaishakh Keshava , Phillip Wallis , Ananth Balashankar , Peter Stone , Lukas Rutishauser

In recent years, agentic artificial intelligence (AI) systems are becoming increasingly widespread. These systems allow agents to use various tools, such as web browsers, compilers, and more. However, despite their popularity, agentic AI…

The rapid pace of recent research in AI has been driven in part by the presence of fast and challenging simulation environments. These environments often take the form of games; with tasks ranging from simple board games, to competitive…

AI agents powered by large language models (LLMs) are being deployed at scale, yet we lack a systematic understanding of how the choice of backbone LLM affects agent security. The non-deterministic sequential nature of AI agents complicates…

Cryptography and Security · Computer Science 2026-02-25 Julia Bazinska , Max Mathys , Francesco Casucci , Mateo Rojas-Carulla , Xander Davies , Alexandra Souly , Niklas Pfister

Autonomous multi-agent systems (MAS) are useful for automating complex tasks but raise trust concerns due to risks such as miscoordination or goal misalignment. Explainability is vital for users' trust calibration, but explainable MAS face…

Artificial Intelligence · Computer Science 2025-10-30 Bálint Gyevnár , Christopher G. Lucas , Stefano V. Albrecht , Shay B. Cohen

Large language model (LLM) agents are rapidly becoming trusted copilots in high-stakes domains like software development and healthcare. However, this deepening trust introduces a novel attack surface: Agent-Mediated Deception (AMD), where…

Human-Computer Interaction · Computer Science 2026-02-25 Xinfeng Li , Shenyu Dai , Kelong Zheng , Yue Xiao , Gelei Deng , Wei Dong , Xiaofeng Wang

The prevalent deployment of Large Language Model agents such as OpenClaw unlocks potential in real-world applications, while amplifying safety concerns. Among these concerns, the self-replication risk of LLM agents driven by objective…

Artificial Intelligence · Computer Science 2026-04-02 Boxuan Zhang , Yi Yu , Jiaxuan Guo , Jing Shao

Multi-agent Large Language Model (LLM) systems have emerged as powerful architectures for complex task decomposition and collaborative problem-solving. However, their long-term behavioral stability remains largely unexamined. This study…

Artificial Intelligence · Computer Science 2026-01-08 Abhishek Rath

In the age of large language models (LLMs), autonomous agents have emerged as a powerful paradigm for achieving general intelligence. These agents dynamically leverage tools, memory, and reasoning capabilities to accomplish user-defined…

Artificial Intelligence · Computer Science 2025-08-05 Chaojia Yu , Zihan Cheng , Hanwen Cui , Yishuo Gao , Zexu Luo , Yijin Wang , Hangbin Zheng , Yong Zhao

This position paper argues that enforcing LLM agent safety within a single abstraction layer is not merely suboptimal but categorically insufficient for deployed LLM agents -- a structural consequence of how agent execution works, not a…

Artificial Intelligence · Computer Science 2026-05-19 S. Bensalem , Y. Dong , M. Franzle , X. Huang , J. Kroger , D. Nickovic , A. Nouri , R. Roy , C. Wu

Security Operations Centers (SOCs) increasingly encounter difficulties in correlating heterogeneous alerts, interpreting multi-stage attack progressions, and selecting safe and effective response actions. This study introduces AgentSOC, a…

Cryptography and Security · Computer Science 2026-04-23 Joyjit Roy , Samaresh Kumar Singh

As the industry increasingly adopts agentic AI systems, understanding their unique vulnerabilities becomes critical. Prior research suggests that security flaws at the model level do not fully capture the risks present in agentic…

Artificial Intelligence · Computer Science 2025-09-23 Ilham Wicaksono , Zekun Wu , Rahul Patel , Theo King , Adriano Koshiyama , Philip Treleaven

Current safety alignment for Large Language Models (LLMs) implicitly optimizes for a "modal adult user," leaving models vulnerable to distributional shifts in user cognition. We present ChildSafe, a benchmark that quantifies alignment…

Computers and Society · Computer Science 2026-01-21 Abhejay Murali , Saleh Afroogh , Kevin Chen , David Atkinson , Amit Dhurandhar , Junfeng Jiao

Large Language Models (LLMs) are increasingly deployed as agentic systems that plan, memorize, and act in open-world environments. This shift brings new security problems: failures are no longer only unsafe text generation, but can become…

Cryptography and Security · Computer Science 2026-03-03 Zhihang Deng , Jiaping Gui , Weinan Zhang

Agentic AI systems, built upon large language models (LLMs) and deployed in multi-agent configurations, are redefining intelligence, autonomy, collaboration, and decision-making across enterprise and societal domains. This review presents a…

Artificial Intelligence · Computer Science 2025-12-19 Shaina Raza , Ranjan Sapkota , Manoj Karkee , Christos Emmanouilidis

Large Language Models (LLMs) have been increasingly integrated into computer-use agents, which can autonomously operate tools on a user's computer to accomplish complex tasks. However, due to the inherently unstable and unpredictable nature…

Cryptography and Security · Computer Science 2025-09-10 Haitao Hu , Peng Chen , Yanpeng Zhao , Yuqi Chen