English
Related papers

Related papers: Misusing Tools in Large Language Models With Visua…

200 papers

Large Language Models (LLMs) are increasingly used in applications where the model selects from competing third-party content, such as in LLM-powered search engines or chatbot plugins. In this paper, we introduce Preference Manipulation…

Cryptography and Security · Computer Science 2024-07-03 Fredrik Nestaas , Edoardo Debenedetti , Florian Tramèr

With the development of large language models (LLMs) like ChatGPT, both their vast applications and potential vulnerabilities have come to the forefront. While developers have integrated multiple safety mechanisms to mitigate their misuse,…

Computation and Language · Computer Science 2024-07-23 Xiao Liu , Liangzhi Li , Tong Xiang , Fuying Ye , Lu Wei , Wangyue Li , Noa Garcia

Multimodal Large Language Models (MLLMs) demonstrate exceptional performance in cross-modality interaction, yet they also suffer adversarial vulnerabilities. In particular, the transferability of adversarial examples remains an ongoing…

Computer Vision and Pattern Recognition · Computer Science 2025-07-22 Hao Cheng , Erjia Xiao , Jiayan Yang , Jinhao Duan , Yichi Wang , Jiahang Cao , Qiang Zhang , Le Yang , Kaidi Xu , Jindong Gu , Renjing Xu

Multimodal Large Language Models (MLLMs) have demonstrated exceptional performance in artificial intelligence by facilitating integrated understanding across diverse modalities, including text, images, video, audio, and speech. However,…

Computer Vision and Pattern Recognition · Computer Science 2025-03-19 Chengze Jiang , Zhuangzhuang Wang , Minjing Dong , Jie Gui

Large language models (LLMs) are complex artificial intelligence systems capable of understanding, generating and translating human language. They learn language patterns by analyzing large amounts of text data, allowing them to perform…

Cryptography and Security · Computer Science 2024-03-15 Biwei Yan , Kun Li , Minghui Xu , Yueyan Dong , Yue Zhang , Zhaochun Ren , Xiuzhen Cheng

We argue that Large language models (LLMs) will soon alter the economics of cyberattacks. Instead of attacking the most commonly used software and monetizing exploits by targeting the lowest common denominator among victims, LLMs enable…

The proliferation of Large Language Models (LLMs) has introduced critical security challenges, where adversarial actors can manipulate input prompts to cause significant harm and circumvent safety alignments. These prompt-based attacks…

We introduce the Adversarial Confusion Attack, a new class of threats against multimodal large language models (MLLMs). Unlike jailbreaks or targeted misclassification, the goal is to induce systematic disruption that makes the model…

Computation and Language · Computer Science 2025-12-02 Jakub Hoscilowicz , Artur Janicki

Growing applications of large language models (LLMs) trained by a third party raise serious concerns on the security vulnerability of LLMs.It has been demonstrated that malicious actors can covertly exploit these vulnerabilities in LLMs…

Cryptography and Security · Computer Science 2023-12-11 Shuli Jiang , Swanand Ravindra Kadhe , Yi Zhou , Ling Cai , Nathalie Baracaldo

This paper investigates the real-world vulnerabilities of audio-based large language models (ALLMs), such as Qwen2-Audio. We first demonstrate that an adversary can craft stealthy audio perturbations to manipulate ALLMs into exhibiting…

Cryptography and Security · Computer Science 2025-07-10 Vinu Sankar Sadasivan , Soheil Feizi , Rajiv Mathews , Lun Wang

Large Language Model (LLM) agents remain vulnerable to safety threats from the external environment, where attackers inject adversarial content into external observations such as tool-returned data, webpages, or MCP context, causing harmful…

Artificial Intelligence · Computer Science 2026-05-28 Yongxiang Li , Moxin Li , Zhixin Ma , Fengbin Zhu , Dongrui Liu , Wenjie Wang , Fuli Feng

Large Language Models (LLMs) are becoming vital tools that help us solve and understand complex problems by acting as digital assistants. LLMs can generate convincing explanations, even when only given the inputs and outputs of these…

Computation and Language · Computer Science 2024-10-14 Rohan Ajwani , Shashidhar Reddy Javaji , Frank Rudzicz , Zining Zhu

Large Language Models (LLMs) have demonstrated extraordinary capabilities and contributed to multiple fields, such as generating and summarizing text, language translation, and question-answering. Nowadays, LLM is becoming a very popular…

Computation and Language · Computer Science 2024-11-18 Badhan Chandra Das , M. Hadi Amini , Yanzhao Wu

Recent research on large language models (LLMs) has demonstrated their ability to understand and employ deceptive behavior, even without explicit prompting. However, such behavior has only been observed in rare, specialized cases and has…

Computation and Language · Computer Science 2025-06-24 Laurène Vaugrante , Francesca Carlon , Maluna Menke , Thilo Hagendorff

Large Language Models (LLMs) are transforming human decision-making by acting as cognitive collaborators. Yet, this promise comes with a paradox: while LLMs can improve accuracy, they may also erode independent reasoning, promote…

Cryptography and Security · Computer Science 2025-09-09 Irdin Pekaric , Philipp Zech , Tom Mattson

The prevalence and strong capability of large language models (LLMs) present significant safety and ethical risks if exploited by malicious users. To prevent the potentially deceptive usage of LLMs, recent works have proposed algorithms to…

Computation and Language · Computer Science 2023-10-20 Zhouxing Shi , Yihan Wang , Fan Yin , Xiangning Chen , Kai-Wei Chang , Cho-Jui Hsieh

Despite various approaches being employed to detect vulnerabilities, the number of reported vulnerabilities shows an upward trend over the years. This suggests the problems are not caught before the code is released, which could be caused…

Cryptography and Security · Computer Science 2025-02-14 Karl Tamberg , Hayretdin Bahsi

Large Language Models (LLMs) can generate content that is as persuasive as human-written text and appear capable of selectively producing deceptive outputs. These capabilities raise concerns about potential misuse and unintended…

Computation and Language · Computer Science 2024-12-24 Cameron R. Jones , Benjamin K. Bergen

Large language models (LLMs) are now routinely used to autonomously execute complex tasks, from natural language processing to dynamic workflows like web searches. The usage of tool-calling and Retrieval Augmented Generation (RAG) allows…

Cryptography and Security · Computer Science 2026-04-13 Dennis Rall , Bernhard Bauer , Mohit Mittal , Thomas Fraunholz

Generative AI, including large language models (LLMs) have the potential -- and already are being used -- to increase the speed, scale, and types of unsafe conversations online. LLMs lower the barrier for entry for bad actors to create…

Human-Computer Interaction · Computer Science 2025-07-31 Owen Hoffman , Kangze Peng , Zehua You , Sajid Kamal , Sukrit Venkatagiri