English
Related papers

Related papers: ClawTrap: A MITM-Based Red-Teaming Framework for R…

200 papers

Code agents powered by large language models can execute shell commands on behalf of users, introducing severe security vulnerabilities. This paper presents a two-phase security analysis of the OpenClaw platform. As an open-source AI agent…

Cryptography and Security · Computer Science 2026-03-12 Zhengyang Shan , Jiayun Xin , Yue Zhang , Minghui Xu

Large language model (LLM) safety evaluation is moving from content moderation to action security as modern systems gain persistent state, tool access, and autonomous control loops. Existing jailbreak frameworks still leave a gap between…

Cryptography and Security · Computer Science 2026-03-20 Yipu Dou , Wang Yang

Automated red-teaming has become a crucial approach for uncovering vulnerabilities in large language models (LLMs). However, most existing methods focus on isolated safety flaws, limiting their ability to adapt to dynamic defenses and…

Cryptography and Security · Computer Science 2025-01-06 Yanjiang Liu , Shuhen Zhou , Yaojie Lu , Huijia Zhu , Weiqiang Wang , Hongyu Lin , Ben He , Xianpei Han , Le Sun

As large language models (LLMs) become increasingly capable, security and safety evaluation are crucial. While current red teaming approaches have made strides in assessing LLM vulnerabilities, they often rely heavily on human input and…

Cryptography and Security · Computer Science 2025-03-21 Andy Zhou , Kevin Wu , Francesco Pinto , Zhaorun Chen , Yi Zeng , Yu Yang , Shuang Yang , Sanmi Koyejo , James Zou , Bo Li

While prior red-teaming efforts have focused on eliciting harmful text outputs from large language models (LLMs), such approaches fail to capture agent-specific vulnerabilities that emerge through multi-step tool execution, particularly in…

Cryptography and Security · Computer Science 2026-03-25 Hyomin Lee , Sangwoo Park , Yumin Choi , Sohyun An , Seanie Lee , Sung Ju Hwang

AI-enabled Security Orchestration, Automation, and Response (SOAR) systems increasingly employ autonomous agents for cyber defense, yet their resilience to adaptive adversaries is underexplored. We introduce an autonomous red teaming…

Cryptography and Security · Computer Science 2026-05-19 Ayan Javeed Shaikh , Nathaniel D. Bastian , Ankit Shah

As large language models grow in capability and agency, identifying vulnerabilities through red-teaming becomes vital for safe deployment. However, traditional prompt-engineering approaches may prove ineffective once red-teaming turns into…

Artificial Intelligence · Computer Science 2026-02-10 Alexander Panfilov , Paul Kassianik , Maksym Andriushchenko , Jonas Geiping

The advancement of Large Language Models (LLMs) has raised concerns regarding their dual-use potential in cybersecurity. Existing evaluation frameworks overwhelmingly focus on Information Technology (IT) environments, failing to capture the…

Cryptography and Security · Computer Science 2026-04-08 Gustav Keppler , Moritz Gstür , Veit Hagenmeyer

Search agents connect LLMs to the Internet, enabling them to access broader and more up-to-date information. However, this also introduces a new threat surface: unreliable search results can mislead agents into producing unsafe outputs.…

Artificial Intelligence · Computer Science 2026-05-29 Jianshuo Dong , Sheng Guo , Hao Wang , Xun Chen , Zhuotao Liu , Tianwei Zhang , Ke Xu , Minlie Huang , Han Qiu

Conventional language model (LM) safety alignment relies on a reactive, disjoint procedure: attackers exploit a static model, followed by defensive fine-tuning to patch exposed vulnerabilities. This sequential approach creates a mismatch --…

Machine Learning · Computer Science 2025-10-07 Mickel Liu , Liwei Jiang , Yancheng Liang , Simon Shaolei Du , Yejin Choi , Tim Althoff , Natasha Jaques

Cybersecurity threats are becoming increasingly sophisticated, making traditional defense mechanisms and manual red teaming approaches insufficient for modern organizations. While red teaming has long been recognized as an effective method…

Cryptography and Security · Computer Science 2026-02-26 Shruti Srivastava , Kiranmayee Janardhan , Shaurya Jauhari

The increasing deployment of large language models (LLMs) in safety-critical applications raises fundamental challenges in systematically evaluating robustness against adversarial behaviors. Existing red-teaming practices are largely manual…

LLMs are increasingly deployed as autonomous agents with access to tools, databases, and external services, yet practitioners (across different sectors) lack systematic methods to assess how known threat classes translate into concrete…

Artificial Intelligence · Computer Science 2026-05-12 Tim Van hamme , Thomas Vissers , Javier Carnerero-Cano , Mario Fritz , Emil C. Lupu , Lieven Desmet , Dinil Mon Divakaran

Extensive efforts have been made before the public release of Large language models (LLMs) to align their behaviors with human values. However, even meticulously aligned LLMs remain vulnerable to malicious manipulations such as…

Cryptography and Security · Computer Science 2024-10-01 Zeguan Xiao , Yan Yang , Guanhua Chen , Yun Chen

Fine-tuning-as-a-service, while commercially successful for Large Language Model (LLM) providers, exposes models to harmful fine-tuning attacks. As a widely explored defense paradigm against such attacks, unlearning attempts to remove…

Cryptography and Security · Computer Science 2025-05-23 Biao Yi , Tiansheng Huang , Baolei Zhang , Tong Li , Lihai Nie , Zheli Liu , Li Shen

The proliferation of jailbreak attacks against large language models (LLMs) highlights the need for robust security measures. However, in multi-round dialogues, malicious intentions may be hidden in interactions, leading LLMs to be more…

Cryptography and Security · Computer Science 2025-05-26 Weiyang Guo , Jing Li , Wenya Wang , YU LI , Daojing He , Jun Yu , Min Zhang

Open agentic systems combine LLM-based planning with external capabilities, persistent memory, and privileged execution. They are used in coding assistants, browser copilots, and enterprise automation. OpenClaw is a visible instance of this…

Cryptography and Security · Computer Science 2026-03-30 Shiping Chen , Qin Wang , Guangsheng Yu , Xu Wang , Liming Zhu

Large Language Models (LLMs) are increasingly integrated into high-stakes applications, making robust safety guarantees a central practical and commercial concern. Existing safety evaluations predominantly rely on fixed collections of…

Computation and Language · Computer Science 2026-03-23 Zafir Shamsi , Nikhil Chekuru , Zachary Guzman , Shivank Garg

Web Agents are increasingly deployed to perform complex tasks in real web environments, yet their security evaluation remains fragmented and difficult to standardize. We present WebTrap Park, an automated platform for systematic security…

Artificial Intelligence · Computer Science 2026-01-14 Xinyi Wu , Jiagui Chen , Geng Hong , Jiayi Dong , Xudong Pan , Jiarun Dai , Min Yang

The rapid evolution of Large Language Models (LLMs) into autonomous, tool-calling agents has fundamentally altered the cybersecurity landscape. Frameworks like OpenClaw grant AI systems operating-system-level permissions and the autonomy to…

Cryptography and Security · Computer Science 2026-03-16 Zonghao Ying , Xiao Yang , Siyang Wu , Yumeng Song , Yang Qu , Hainan Li , Tianlin Li , Jiakai Wang , Aishan Liu , Xianglong Liu