English
Related papers

Related papers: What Makes a Good LLM Agent for Real-world Penetra…

200 papers

Penetration testing is essential for identifying vulnerabilities in web applications before real adversaries can exploit them. Recent work has explored automating this process with Large Language Model (LLM)-powered agents, but existing…

Software Engineering · Computer Science 2026-01-13 Huihui Huang , Jieke Shi , Junkai Chen , Ting Zhang , Yikun Li , Chengran Yang , Eng Lieh Ouh , Lwin Khin Shar , David Lo

This work introduces xOffense, an AI-driven, multi-agent penetration testing framework that shifts the process from labor-intensive, expert-driven manual efforts to fully automated, machine-executable workflows capable of scaling seamlessly…

Cryptography and Security · Computer Science 2026-04-28 Phung Duc Luong , Le Tran Gia Bao , Nguyen Vu Khai Tam , Dong Huu Nguyen Khoa , Nguyen Huu Quyen , Van-Hau Pham , Phan The Duy

Although LLM-based agents, powered by Large Language Models (LLMs), can use external tools and memory mechanisms to solve complex real-world tasks, they may also introduce critical security vulnerabilities. However, the existing literature…

Cryptography and Security · Computer Science 2025-06-02 Hanrong Zhang , Jingyuan Huang , Kai Mei , Yifei Yao , Zhenting Wang , Chenlu Zhan , Hongwei Wang , Yongfeng Zhang

Generative AI agents, software systems powered by Large Language Models (LLMs), are emerging as a promising approach to automate cybersecurity tasks. Among the others, penetration testing is a challenging field due to the task complexity…

Cryptography and Security · Computer Science 2024-10-29 Luca Gioacchini , Marco Mellia , Idilio Drago , Alexander Delsanto , Giuseppe Siracusano , Roberto Bifulco

Autonomous agents have recently achieved remarkable progress across diverse domains, yet most evaluations focus on short-horizon, fully observable tasks. In contrast, many critical real-world tasks, such as large-scale software development,…

Penetration testing is essential to securing modern web infrastructures, yet traditional manual methods struggle to keep pace with their scale and complexity. Large Language Models (LLMs) offer new opportunities for automating these tasks,…

Cryptography and Security · Computer Science 2026-05-26 William Guanting Li , Alsharif Abuadbba , Kristen Moore , Dan Dongseong Kim

Autonomous agents based on Large Language Models (LLMs) are increasingly being utilized in complex software systems. However, reliability remains a significant challenge due to unpredictable failures such as hallucinations, execution…

Software Engineering · Computer Science 2026-05-11 Cheonsu Jeong , Younggun Shin

As the frequency of cyber threats increases, conventional penetration testing is failing to capture the entirety of todays complex environments. To solve this problem, we propose the Vulnerability Mitigation System (VMS), a novel agent…

Cryptography and Security · Computer Science 2025-07-30 Farzana Abdulzada

Recently, autonomous agents built on large language models (LLMs) have experienced significant development and are being deployed in real-world applications. These agents can extend the base LLM's capabilities in multiple ways. For example,…

Cryptography and Security · Computer Science 2024-07-31 Boyang Zhang , Yicong Tan , Yun Shen , Ahmed Salem , Michael Backes , Savvas Zannettou , Yang Zhang

LLM agents are increasingly deployed in long-horizon, complex environments to solve challenging problems, but this expansion exposes them to long-horizon attacks that exploit multi-turn user-agent-environment interactions to achieve…

Artificial Intelligence · Computer Science 2026-02-20 Tanqiu Jiang , Yuhui Wang , Jiacheng Liang , Ting Wang

Large Language Model (LLM) agents increasingly act through external tools, making their safety contingent on tool-call workflows rather than text generation alone. While recent benchmarks evaluate agents across diverse environments and risk…

Software Engineering · Computer Science 2026-03-20 Xuan Chen , Lu Yan , Ruqi Zhang , Xiangyu Zhang

The evolution of Large Language Models (LLMs) from passive text processors to autonomous agents has established planning as a core component of modern intelligence. However, achieving generalized planning remains elusive, not only by the…

Large language models (LLMs) have evolved into agentic systems capable of autonomous tool use and multi-step reasoning for complex problem-solving. However, post-training approaches building upon general-purpose foundation models…

As autonomous agents (e.g., OpenClaw) increasingly operate with deep system-level privileges to execute complex tasks, they introduce severe, unmitigated security risks. Current vulnerability analyses overwhelmingly focus on single-turn,…

Cryptography and Security · Computer Science 2026-05-22 Jianan Ma , Xiaohu Du , Ruixiao Lin , Yaoxiang Bian , Jialuo Chen , Jingyi Wang , Xiaofang Yang , Shiwen Cui , Changhua Meng , Xinhao Deng , Zhen Wang

Penetration testing is essential to ensure Web security, which can detect and fix vulnerabilities in advance, and prevent data leakage and serious consequences. The powerful inference capabilities of large language models (LLMs) have made…

Cryptography and Security · Computer Science 2024-11-05 Benlong Wu , Guoqiang Chen , Kejiang Chen , Xiuwei Shang , Jiapeng Han , Yanru He , Weiming Zhang , Nenghai Yu

The integration of tool use into large language models (LLMs) enables agentic systems with real-world impact. In the meantime, unlike standalone LLMs, compromised agents can execute malicious workflows with more consequential impact,…

Cryptography and Security · Computer Science 2025-02-17 Jizhou Chen , Samuel Lee Cong

Penetration testing, a crucial industrial practice for ensuring system security, has traditionally resisted automation due to the extensive expertise required by human professionals. Large Language Models (LLMs) have shown significant…

Software Engineering · Computer Science 2024-06-04 Gelei Deng , Yi Liu , Víctor Mayoral-Vilches , Peng Liu , Yuekang Li , Yuan Xu , Tianwei Zhang , Yang Liu , Martin Pinzger , Stefan Rass

This paper presents a critical examination of the surprising efficacy of Large Language Models (LLMs) in penetration testing. The paper thoroughly reviews the evolution of LLMs and their rapidly expanding capabilities which render them…

Cryptography and Security · Computer Science 2025-07-02 Andreas Happe , Jürgen Cito

Web applications remain the dominant attack surface in cybersecurity, where vulnerabilities such as SQL injection, XSS, and business logic flaws continue to cause significant data breaches. While penetration testing is effective for…

Cryptography and Security · Computer Science 2026-03-31 Tran Vy Khang , Nguyen Dang Nguyen Khang , Nghi Hoang Khoa , Do Thi Thu Hien , Van-Hau Pham , Phan The Duy

Developing autonomous agents for web-based tasks is a core challenge in AI. While Large Language Model (LLM) agents can interpret complex user requests, they often operate as black boxes, making it difficult to diagnose why they fail or how…

Artificial Intelligence · Computer Science 2026-03-16 Orit Shahnovsky , Rotem Dror