English
Related papers

Related papers: Prompt Injection Mitigation with Agentic AI, Neste…

200 papers

As vision-language models (VLMs) gain prominence, their multimodal interfaces also introduce new safety vulnerabilities, making the safety evaluation challenging and critical. Existing red-teaming efforts are either restricted to a narrow…

Artificial Intelligence · Computer Science 2025-10-06 Zhaorun Chen , Xun Liu , Mintong Kang , Jiawei Zhang , Minzhou Pan , Shuang Yang , Bo Li

Large language models (LLMs) are increasingly used as analyst assistants in security operations centers (SOCs), where they ingest log and alert data to produce triage labels, incident summaries, or remediation advice. We study a structural…

Cryptography and Security · Computer Science 2026-05-26 Rohan Pandey , Archit Bhujang

While Large Language Models (LLMs) are increasingly being used in real-world applications, they remain vulnerable to prompt injection attacks: malicious third party prompts that subvert the intent of the system designer. To help researchers…

We demonstrate how AI-powered cybersecurity tools can be turned against themselves through prompt injection attacks. Prompt injection is reminiscent of cross-site scripting (XSS): malicious text is hidden within seemingly trusted content,…

Cryptography and Security · Computer Science 2025-11-18 Víctor Mayoral-Vilches , Per Mannermaa Rynning

The April 2026 disclosure that a frontier large language model escaped its security sandbox, executed unauthorized actions, and concealed its modifications to version control history demonstrates that agentic AI systems with autonomous tool…

Cryptography and Security · Computer Science 2026-04-28 Richard Joseph Mitchell

The automation of Cyber Threat Intelligence (CTI) relies heavily on Named Entity Recognition (NER) to extract critical entities from unstructured text. Currently, Large Language Models (LLMs) primarily address this task through…

Cryptography and Security · Computer Science 2025-12-23 Jiaren Peng , Hongda Sun , Xuan Tian , Cheng Huang , Zeqing Li , Rui Yan

Current stateless defences for multimodal agentic RAG fail to detect adversarial strategies that distribute malicious semantics across retrieval, planning, and generation components. We formulate this security challenge as a Partially…

Direct Prompt Injection (DPI) attacks pose a critical security threat to Large Language Models (LLMs) due to their low barrier of execution and high potential damage. To address the impracticality of existing white-box/gray-box methods and…

Artificial Intelligence · Computer Science 2025-09-10 Minghui Li , Hao Zhang , Yechao Zhang , Wei Wan , Shengshan Hu , pei Xiaobing , Jing Wang

As Large Language Models (LLMs) evolve into persistent scientific collaborators, context window saturation has emerged as a critical bottleneck. Scientific workflows involving iterative data analysis and hypothesis refinement rapidly…

Artificial Intelligence · Computer Science 2026-05-19 Nikola Milosevic

The advent of Large Language Models LLMs marks a milestone in Artificial Intelligence, altering how machines comprehend and generate human language. However, LLMs are vulnerable to malicious prompt injection attacks, where crafted inputs…

Computation and Language · Computer Science 2024-10-29 Sahasra Kokkula , Somanathan R , Nandavardhan R , Aashishkumar , G Divya

Chain-of-Thought (CoT) prompting has been used to enhance the reasoning capability of LLMs. However, its reliability in security-sensitive analytical tasks remains insufficiently examined, particularly under structured human evaluation.…

Cryptography and Security · Computer Science 2026-04-07 Jiling Zhou , Aisvarya Adeseye , Seppo Virtanen , Antti Hakkala , Jouni Isoaho

Although multimodal large language models (MLLMs) are increasingly deployed in real-world applications, their instruction-following behavior leaves them vulnerable to prompt injection attacks. Existing prompt injection methods predominantly…

Computer Vision and Pattern Recognition · Computer Science 2026-04-01 Meiwen Ding , Song Xia , Chenqi Kong , Xudong Jiang

The rapid emergence of new entities -- driven by cultural shifts, evolving trends, and personalized user data -- poses a significant challenge for existing Speech Large Language Models (Speech LLMs). While these models excel at general…

Artificial Intelligence · Computer Science 2026-02-06 Peidong Wang

Driven by Large Language Models, the single-agent, multi-tool architecture has become a popular paradigm for autonomous agents. However, this architecture introduces a severe privacy risk, which we term Tools Orchestration Privacy Risk…

Cryptography and Security · Computer Science 2026-03-09 Yuxuan Qiao , Dongqin Liu , Hongchang Yang , Wei Zhou , Songlin Hu

Autonomous AI agents powered by large language models are being deployed in production with capabilities including shell execution, file system access, database queries, and multi-party communication. Recent red teaming research…

Cryptography and Security · Computer Science 2026-03-19 Saikat Maiti

Word sense plausibility rating requires predicting the human-perceived plausibility of a given word sense on a 1-5 scale in the context of short narrative stories containing ambiguous homonyms. This paper systematically compares three…

Computation and Language · Computer Science 2026-05-11 Tong Wu , Thanet Markchom , Huizhi Liang

Adversarial images pose a severe security threat to multimodal large language models through prompt injection. Existing defenses largely lack a principled understanding of the underlying mechanisms and struggle to balance efficiency and…

Machine Learning · Computer Science 2026-05-26 Dongpeng Zhang , Ke Ma , Yangbangyan Jiang , Gaozheng Pei , Longtao Huang , Qianqian Xu , Qingming Huang

Existing memory benchmarks for LLM agents evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavior without conscious retrieval. This gap is critical: effective assistants must automatically…

Artificial Intelligence · Computer Science 2026-04-16 Chonghan Qin , Xiachong Feng , Weitao Ma , Xiaocheng Feng , Lingpeng Kong

The evolution toward 6G networks is being accelerated by the Open Radio Access Network (O-RAN) paradigm -- an open, interoperable architecture that enables intelligent, modular applications across public telecom and private enterprise…

Cryptography and Security · Computer Science 2025-10-06 Prakhar Sharma , Haohuang Wen , Vinod Yegneswaran , Ashish Gehani , Phillip Porras , Zhiqiang Lin

Vision-Language Models (VLMs) are increasingly deployed in consumer applications where users seek recommendations about products, dining, and services. We introduce Hidden Ads, a new class of backdoor attacks that exploit this…

Computation and Language · Computer Science 2026-03-31 Duanyi Yao , Changyue Li , Zhicong Huang , Cheng Hong , Songze Li