English
Related papers

Related papers: When Think-with-Image Meets Safety: What Determine…

200 papers

Cross-view spatial reasoning remains a weak spot for vision-language models (VLMs): they often reason in language and lose the fine-grained geometry needed for the task. Thinking with images aims to address this by generating an…

Computer Vision and Pattern Recognition · Computer Science 2026-05-27 Qian Yang , Ankur Sikarwar , Huy Le , Le Zhang , Zhuan Shi , Perouz Taslakian , Aishwarya Agrawal

Large language models exhibit safety degradation in non-English languages. Standard evaluation relies on Jailbreak Success Rate (JSR), which confounds several safety-driving factors into one, obscuring the specific cause(s) of safety…

Computation and Language · Computer Science 2026-05-19 Max Zhang , Ameen Patel , Sang T. Truong , Sanmi Koyejo

Despite inheriting security measures from underlying language models, Vision-Language Models (VLMs) may still be vulnerable to safety alignment issues. Through empirical analysis, we uncover two critical findings: scenario-matched images…

Computer Vision and Pattern Recognition · Computer Science 2024-12-02 Shuyang Hao , Bryan Hooi , Jun Liu , Kai-Wei Chang , Zi Huang , Yujun Cai

In the realm of large vision language models (LVLMs), jailbreak attacks serve as a red-teaming approach to bypass guardrails and uncover safety implications. Existing jailbreaks predominantly focus on the visual modality, perturbing solely…

Computer Vision and Pattern Recognition · Computer Science 2024-07-02 Zonghao Ying , Aishan Liu , Tianyuan Zhang , Zhengmin Yu , Siyuan Liang , Xianglong Liu , Dacheng Tao

Vision-language models remain susceptible to multimodal jailbreaks and over-refusal because safety hinges on both visual evidence and user intent, while many alignment pipelines supervise only the final response. To address this, we present…

Machine Learning · Computer Science 2026-03-04 Zixuan Xu , Tiancheng He , Huahui Yi , Kun Wang , Xi Chen , Gongli Xi , Qiankun Li , Kang Li , Yang Liu , Zhigang Zeng

Vision-Language Models (VLMs) have achieved remarkable progress in multimodal reasoning tasks through enhanced chain-of-thought capabilities. However, this advancement also introduces novel safety risks, as these models become increasingly…

Computer Vision and Pattern Recognition · Computer Science 2025-10-22 Yinan Xia , Yilei Jiang , Yingshui Tan , Xiaoyong Zhu , Xiangyu Yue , Bo Zheng

With the rapid advancement of large language models (LLMs), the safety of LLMs has become a critical concern. Despite significant efforts in safety alignment, current LLMs remain vulnerable to jailbreaking attacks. However, the root causes…

Artificial Intelligence · Computer Science 2026-03-10 Yonghong Deng , Zhen Yang , Ping Jian , Xinyue Zhang , Zhongbin Guo , Chengzhi Li

Multimodal large language models (MLLMs) comprise of both visual and textual modalities to process vision language tasks. However, MLLMs are vulnerable to security-related issues, such as jailbreak attacks that alter the model's input to…

Cryptography and Security · Computer Science 2025-10-27 Xingwei Zhong , Kar Wai Fok , Vrizlynn L. L. Thing

Large Vision Language Models (LVLMs) demonstrate strong capabilities in multimodal reasoning and many real-world applications, such as visual question answering. However, LVLMs are highly vulnerable to jailbreaking attacks. This paper…

Artificial Intelligence · Computer Science 2025-11-19 Badhan Chandra Das , Md Tasnim Jawad , Md Jueal Mia , M. Hadi Amini , Yanzhao Wu

Multimodal Large Language Models (MLLMs) have become widely deployed, yet their safety alignment remains fragile under adversarial inputs. Previous work has shown that increasing inference steps can disrupt safety mechanisms and lead MLLMs…

Computer Vision and Pattern Recognition · Computer Science 2026-04-17 Xiangdong Hu , Yangyang Jiang , Qin Hu , Xiaojun Jia

Multimodal large language models (MLLMs) excel in vision-language tasks but also pose significant risks of generating harmful content, particularly through jailbreak attacks. Jailbreak attacks refer to intentional manipulations that bypass…

Cryptography and Security · Computer Science 2025-07-18 Yi Nian , Shenzhe Zhu , Yuehan Qin , Li Li , Ziyi Wang , Chaowei Xiao , Yue Zhao

Reinforcement learning (RL) based post-training for explicit chain-of-thought (e.g., GRPO) improves the reasoning ability of multimodal large-scale reasoning models (MLRMs). But recent evidence shows that it can simultaneously degrade…

Computation and Language · Computer Science 2026-02-12 Soumya Suvra Ghosal , Souradip Chakraborty , Vaibhav Singh , Furong Huang , Dinesh Manocha , Amrit Singh Bedi

There has been an increasing interest in the alignment of large language models (LLMs) with human values. However, the safety issues of their integration with a vision module, or vision language models (VLMs), remain relatively…

Computer Vision and Pattern Recognition · Computer Science 2025-09-30 Xijia Tao , Shuai Zhong , Lei Li , Qi Liu , Lingpeng Kong

Large Multimodal Reasoning Models (LMRMs) are moving into real applications, where they must be both useful and safe. Safety is especially challenging in multimodal settings: images and text can be combined to bypass guardrails, and single…

Artificial Intelligence · Computer Science 2025-10-07 Yizhuo Ding , Mingkang Chen , Qiuhua Liu , Fenghua Weng , Wanying Qu , Yue Yang , Yugang Jiang , Zuxuan Wu , Yanwei Fu , Wenqi Shao

Understanding the vulnerabilities of Large Vision Language Models (LVLMs) to jailbreak attacks is essential for their responsible real-world deployment. Most previous work requires access to model gradients, or is based on human knowledge…

Cryptography and Security · Computer Science 2025-09-03 Ruoxi Cheng , Yizhong Ding , Shuirong Cao , Ranjie Duan , Xiaoshuang Jia , Shaowei Yuan , Simeng Qin , Zhiqiang Wang , Xiaojun Jia

Vision-Language Models (VLMs) have achieved remarkable performance on a variety of tasks, yet they remain vulnerable to jailbreak attacks that compromise safety and reliability. In this paper, we provide an information-theoretic framework…

Machine Learning · Computer Science 2025-02-04 Ching-Chia Kao , Chia-Mu Yu , Chun-Shien Lu , Chu-Song Chen

While Multimodal Large Language Models (MLLMs) show remarkable capabilities, their safety alignments are susceptible to jailbreak attacks. Existing attack methods typically focus on text-image interplay, treating the visual modality as a…

Computer Vision and Pattern Recognition · Computer Science 2025-12-03 Yuan Xiong , Ziqi Miao , Lijun Li , Chen Qian , Jie Li , Jing Shao

While recent advancements in large language model (LLM) alignment have enabled the effective identification of malicious objectives involving scene nesting and keyword rewriting, our study reveals that these methods remain inadequate at…

Computation and Language · Computer Science 2024-10-10 Tianyu Wu , Lingrui Mei , Ruibin Yuan , Lujun Li , Wei Xue , Yike Guo

We present MultiBreak, a scalable and diverse multi-turn jailbreak benchmark to evaluate large language model (LLM) safety. Multi-turn jailbreaks mimic natural conversational settings, making them easier to bypass safety-aligned LLM than…

Computation and Language · Computer Science 2026-05-05 Jialin Song , Xiaodong Liu , Weiwei Yang , Wuyang Chen , Mingqian Feng , Xuekai Zhu , Jianfeng Gao

Text-to-Image models may generate harmful content, such as pornographic images, particularly when unsafe prompts are submitted. To address this issue, safety filters are often added on top of text-to-image models, or the models themselves…

Cryptography and Security · Computer Science 2026-01-09 Zhengyuan Jiang , Yuepeng Hu , Yuchen Yang , Yinzhi Cao , Neil Zhenqiang Gong