English
Related papers

Related papers: Trust No Tool: Evaluating and Defending LLM Agents…

200 papers

AI control protocols serve as a defense mechanism to stop untrusted LLM agents from causing harm in autonomous settings. Prior work treats this as a security problem, stress testing with exploits that use the deployment context to subtly…

Autonomous agents powered by large language models (LLMs) enable novel use cases in domains where responsible action is increasingly important. Yet the inherent unpredictability of LLMs raises safety concerns about agent reliability. In…

Artificial Intelligence · Computer Science 2025-05-19 Jan Chojnacki

By providing a standardized interface for LLM agents to interact with external tools, the Model Context Protocol (MCP) is quickly becoming a cornerstone of the modern autonomous agent ecosystem. However, it creates novel attack surfaces due…

Cryptography and Security · Computer Science 2025-08-22 Zhiqiang Wang , Yichao Gao , Yanting Wang , Suyuan Liu , Haifeng Sun , Haoran Cheng , Guanquan Shi , Haohua Du , Xiangyang Li

Large language model (LLM) systems increasingly power everyday AI applications such as chatbots, computer-use assistants, and autonomous robots, where performance often depends on manually well-crafted prompts. LLM-based prompt optimizers…

Machine Learning · Computer Science 2026-01-14 Andrew Zhao , Reshmi Ghosh , Vitor Carvalho , Emily Lawton , Keegan Hines , Gao Huang , Jack W. Stokes

LLM-based user simulation is the primary mechanism for end-to-end agent evaluation, yet simulated users are poor proxies for real humans: unconstrained LLM defaults produce a Formalism Ceiling (style match rates of 6-8% against real users),…

Human-Computer Interaction · Computer Science 2026-05-21 Ming Zhu , Juntao Tan , Rithesh Murthy , Jielin Qiu , Liangwei Yang , Wenting Zhao , Silvio Savarese , Shelby Heinecke , Huan Wang

Vision-Language Models (VLMs) have demonstrated strong capabilities in aligning visual and textual modalities, enabling a wide range of applications in multimodal understanding and generation. While they excel in zero-shot and transfer…

Computer Vision and Pattern Recognition · Computer Science 2025-09-25 Hao Dong , Moru Liu , Jian Liang , Eleni Chatzi , Olga Fink

LLM agents are evolving rapidly, powered by code execution, tools, and the recently introduced agent skills feature. Skills allow users to extend LLM applications with specialized third-party code, knowledge, and instructions. Although this…

Cryptography and Security · Computer Science 2026-02-26 David Schmotz , Luca Beurer-Kellner , Sahar Abdelnabi , Maksym Andriushchenko

Existing agent-safety evaluation has focused mainly on externally induced risks. Yet agents may still enter unsafe trajectories under benign conditions. We study this complementary but underexplored setting through the lens of…

Machine Learning · Computer Science 2026-04-16 Jiacheng Wang , Jinchang Hou , Fabian Wang , Ping Jian , Chenfu Bao , Zhonghou Lv

LLM-based agents have recently attracted significant attention due to their ability to autonomously invoke relevant tools to accomplish complex tasks. However, recent studies have shown that these agents face severe security risks, which…

Cryptography and Security · Computer Science 2026-05-28 Jiaqi Luo , Songyang Peng , Jiarun Dai , Zhile Chen , Zhuoxiang Shen , Geng Hong , Xudong Pan , Yuan Zhang , Min Yang

The rapid progress in open-source Large Language Models (LLMs) is significantly driving AI development forward. However, there is still a limited understanding of their trustworthiness. Deploying these models at scale without sufficient…

Computation and Language · Computer Science 2024-04-03 Lingbo Mo , Boshi Wang , Muhao Chen , Huan Sun

Large language models (LLMs) are increasingly deployed as agents with access to executable tools, enabling direct interaction with external systems. However, most safety evaluations remain text-centric and assume that compliant language…

Software Engineering · Computer Science 2026-03-24 Shasha Yu , Fiona Carroll , Barry L. Bentley

Indirect prompt injection attacks (IPIAs), where large language models (LLMs) follow malicious instructions hidden in input data, pose a critical threat to LLM-powered agents. In this paper, we present IntentGuard, a general defense…

Cryptography and Security · Computer Science 2025-12-02 Mintong Kang , Chong Xiang , Sanjay Kariyappa , Chaowei Xiao , Bo Li , Edward Suh

Web agents powered by vision-language models (VLMs) enable autonomous interaction with web environments by perceiving and acting on both visual and textual webpage content to accomplish user-specified tasks. However, they are highly…

Cryptography and Security · Computer Science 2026-04-15 Yulin Chen , Tri Cao , Haoran Li , Yue Liu , Yibo Li , Yufei He , Le Minh Khoi , Yangqiu Song , Shuicheng Yan , Bryan Hooi

Even when a tool is explicitly described as unfair and harmful to others, ostensibly safety-aligned LLM agents still voluntarily engage in secret collusion whenever doing so confers a strategic advantage. To investigate this phenomenon, we…

Artificial Intelligence · Computer Science 2026-05-28 Xijie Zeng , Frank Rudzicz

The Model Context Protocol (MCP) is increasingly adopted to standardize the interaction between LLM agents and external tools. However, this trend introduces a new threat: Tool Poisoning Attacks (TPA), where tool metadata is poisoned to…

Cryptography and Security · Computer Science 2026-01-16 Zhiqiang Wang , Haohua Du , Guanquan Shi , Junyang Zhang , HaoRan Cheng , Yunhao Yao , Kaiwen Guo , Xiang-Yang Li

Autonomous agents powered by large language models introduce a class of execution-layer vulnerabilities -- prompt injection, retrieval poisoning, and uncontrolled tool invocation -- that existing guardrails fail to address systematically.…

Cryptography and Security · Computer Science 2026-03-11 Yuxu Ge

Large language model (LLM) agents with extended autonomy unlock new capabilities, but also introduce heightened challenges for LLM safety. In particular, an LLM agent may pursue objectives that deviate from human values and ethical norms, a…

Computation and Language · Computer Science 2026-01-27 Chen Chen , Kim Young Il , Yuan Yang , Wenhao Su , Yilin Zhang , Xueluan Gong , Qian Wang , Yongsen Zheng , Ziyao Liu , Kwok-Yan Lam

As artificial intelligence (AI) assistants become more widely adopted in safety-critical domains, it becomes important to develop safeguards against potential failures or adversarial attacks. A key prerequisite to developing these…

Human-Computer Interaction · Computer Science 2025-04-04 Abed Kareem Musaffar , Anand Gokhale , Sirui Zeng , Rasta Tadayon , Xifeng Yan , Ambuj Singh , Francesco Bullo

The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a new category of safety risk beyond content safety: behavior jailbreak, where an adversary induces an agent to execute dangerous…

Cryptography and Security · Computer Science 2026-05-12 Chiyu Zhang , Huiqin Yang , Bendong Jiang , Xiaolei Zhang , Yiran Zhao , Ruyi Chen , Lu Zhou , Xiaogang Xu , Jiafei Wu , Liming Fang , Zhe Liu

Educational LLM tutors face a core AI alignment challenge: they must follow user intent while preserving pedagogical constraints and safety policies. We present an evaluation methodology for prompt-injection defenses in this setting,…

Cryptography and Security · Computer Science 2026-05-22 Alexandre Cristovão Maiorano