English
Related papers

Related papers: Dynamic Token Reweighting for Robust Vision-Langua…

200 papers

Large language models (LLMs) are shown to be vulnerable to jailbreaking attacks where adversarial prompts are designed to elicit harmful responses. While existing defenses effectively mitigate single-turn attacks by detecting and filtering…

Computation and Language · Computer Science 2026-02-17 Hanjiang Hu , Alexander Robey , Changliu Liu

Large Vision-Language Models (LVLMs) rely on attention-based retrieval of safety instructions to maintain alignment during generation. Existing attacks typically optimize image perturbations to maximize harmful output likelihood, but suffer…

Computer Vision and Pattern Recognition · Computer Science 2026-04-14 Jingru Li , Wei Ren , Tianqing Zhu

Autonomous vehicles rely on deep neural networks (DNNs) for traffic sign recognition, lane centering, and vehicle detection, yet these models are vulnerable to attacks that induce misclassification and threaten safety. Existing defenses…

Computer Vision and Pattern Recognition · Computer Science 2026-02-10 Pedram MohajerAnsari , Amir Salarpour , Michael Kühr , Siyu Huang , Mohammad Hamad , Sebastian Steinhorst , Habeeb Olufowobi , Bing Li , Mert D. Pesé

Large Language Models (LLMs) remain susceptible to jailbreak exploits that bypass safety filters and induce harmful or unethical behavior. This work presents a systematic taxonomy of existing jailbreak defenses across prompt-level,…

Cryptography and Security · Computer Science 2025-11-25 Ryan Wong , Hosea David Yu Fei Ng , Dhananjai Sharma , Glenn Jun Jie Ng , Kavishvaran Srinivasan

Intent-obfuscation-based jailbreak attacks on multimodal large language models (MLLMs) transform a harmful query into a concealed multimodal input to bypass safety mechanisms. We show that such attacks are governed by a…

Artificial Intelligence · Computer Science 2026-05-08 Md Farhamdur Reza , Richeng Jin , Tianfu Wu , Huaiyu Dai

Large language models (LLMs) have achieved remarkable success across diverse applications but remain vulnerable to jailbreak attacks, where attackers craft prompts that bypass safety alignment and elicit unsafe responses. Among existing…

Computation and Language · Computer Science 2026-03-04 Zhi Xu , Jiaqi Li , Xiaotong Zhang , Hong Yu , Han Liu

Multimodal Large Language Models (MLLMs) bridge the gap between visual and textual data, enabling a range of advanced applications. However, complex internal interactions among visual elements and their alignment with text can introduce…

Computer Vision and Pattern Recognition · Computer Science 2025-06-18 Zuopeng Yang , Jiluan Fan , Anli Yan , Erdun Gao , Xin Lin , Tao Li , Kanghua Mo , Changyu Dong

Vision Language Models (VLMs) have demonstrated impressive capabilities in integrating visual and textual information for understanding and reasoning, but remain highly vulnerable to adversarial attacks. While activation steering has…

Computer Vision and Pattern Recognition · Computer Science 2025-09-03 Sihao Wu , Gaojie Jin , Wei Huang , Jianhong Wang , Xiaowei Huang

Vision-Language Models (VLMs) exhibit impressive performance, yet the integration of powerful vision encoders has significantly broadened their attack surface, rendering them increasingly susceptible to jailbreak attacks. However, lacking…

Computer Vision and Pattern Recognition · Computer Science 2026-02-26 Jiaxin Song , Yixu Wang , Jie Li , Rui Yu , Yan Teng , Xingjun Ma , Yingchun Wang

Large Language Models (LLMs) remain vulnerable to jailbreak attacks that bypass their safety mechanisms. Existing attack methods are fixed or specifically tailored for certain models and cannot flexibly adjust attack strength, which is…

Cryptography and Security · Computer Science 2024-10-08 Yiting Dong , Guobin Shen , Dongcheng Zhao , Xiang He , Yi Zeng

Multimodal large language models (MLLMs) are gaining increasing attention. Due to the heterogeneity of their input features, they face significant challenges in terms of jailbreak defenses. Current defense methods rely on costly fine-tuning…

Artificial Intelligence · Computer Science 2026-05-13 Xinyi Zeng , Xue Yang , Jingyuan Zhang , Huanqian Yan , Xiang Chen , Kaiwen Wei , Hankun Kang , Yu Tian

Large Language Models (LLMs) have transformed artificial intelligence by advancing natural language understanding and generation, enabling applications across fields beyond healthcare, software engineering, and conversational systems.…

Despite advances in AI alignment, large language models (LLMs) remain vulnerable to adversarial attacks or jailbreaking, in which adversaries can modify prompts to induce unwanted behavior. While some defenses have been proposed, they have…

Machine Learning · Computer Science 2024-11-11 Andy Zhou , Bo Li , Haohan Wang

Vision-Language Models (VLMs) have emerged as a critical and fast-growing extension of Large Language Models (LLMs) that enable multimodal reasoning through both text and image inputs. Although VLMs enrich the capabilities of language…

Computer Vision and Pattern Recognition · Computer Science 2026-05-19 Yingbing Huang , Tharun Adithya Srikrishnan , Steven K. Reinhardt , Deming Chen

Diffusion Language Models (DLMs) have been seen as a promising competitor for autoregressive language models. However, diffusion language models have long been constrained by slow inference. A core challenge is that their non-autoregressive…

Computation and Language · Computer Science 2025-05-22 Xinyin Ma , Runpeng Yu , Gongfan Fang , Xinchao Wang

In the realm of large vision-language models (LVLMs), adversarial jailbreak attacks serve as a red-teaming approach to identify safety vulnerabilities of these models and their associated defense mechanisms. However, we identify a critical…

Computer Vision and Pattern Recognition · Computer Science 2025-03-18 Shuyang Hao , Yiwei Wang , Bryan Hooi , Jun Liu , Muhao Chen , Zi Huang , Yujun Cai

Jailbreak attacks pose significant threats to large language models (LLMs), enabling attackers to bypass safeguards. However, existing reactive defense approaches struggle to keep up with the rapidly evolving multi-turn jailbreaks, where…

Cryptography and Security · Computer Science 2026-01-08 Siyuan Li , Xi Lin , Jun Wu , Zehao Liu , Haoyu Li , Tianjie Ju , Xiang Chen , Jianhua Li

The rapid progress of Large Language Models (LLMs) has opened up new opportunities across various domains and applications; yet it also presents challenges related to potential misuse. To mitigate such risks, red teaming has been employed…

Cryptography and Security · Computer Science 2025-06-10 Yifan Jiang , Kriti Aggarwal , Tanmay Laud , Kashif Munir , Jay Pujara , Subhabrata Mukherjee

Large language models remain vulnerable to jailbreak attacks, and single-layer defenses often trade security for usability. We present TRYLOCK, the first defense-in-depth architecture that combines four heterogeneous mechanisms across the…

Cryptography and Security · Computer Science 2026-01-08 Scott Thornton

Vision-language models (VLMs) demonstrate strong multimodal capabilities but have been found to be more susceptible to generating harmful content compared to their backbone large language models (LLMs). Our investigation reveals that the…

Machine Learning · Computer Science 2025-01-29 Qing Li , Jiahui Geng , Zongxiong Chen , Kun Song , Lei Ma , Fakhri Karray