English
Related papers

Related papers: MSR-Align: Policy-Grounded Multimodal Alignment fo…

200 papers

Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal perception and generation, yet their safety alignment remains a critical challenge.Existing defenses and vulnerable to multimodal jailbreaks, as visual…

Artificial Intelligence · Computer Science 2025-10-21 MingSheng Li , Guangze Zhao , Sichen Liu

Safeguarding vision-language models (VLMs) is a critical challenge, as existing methods often suffer from over-defense, which harms utility, or rely on shallow alignment, failing to detect complex threats that require deep reasoning. To…

Cryptography and Security · Computer Science 2026-04-03 Nanxi Li , Zhengyue Zhao , G. Edward Suh , Marco Pavone , Chaowei Xiao

Multimodal Large Reasoning Models (MLRMs) demonstrate impressive cross-modal reasoning but often amplify safety risks under adversarial or unsafe prompts, a phenomenon we call the \textit{Reasoning Tax}. Existing defenses mainly act at the…

Machine Learning · Computer Science 2025-10-10 Huahui Yi , Kun Wang , Qiankun Li , Miao Yu , Liang Lin , Gongli Xi , Hao Wu , Xuming Hu , Kang Li , Yang Liu

Multimodal large language models (MLLMs) have demonstrated impressive reasoning and instruction-following capabilities, yet their expanded modality space introduces new compositional safety risks that emerge from complex text-image…

Cryptography and Security · Computer Science 2025-11-18 Xuankun Rong , Wenke Huang , Tingfeng Wang , Daiguo Zhou , Bo Du , Mang Ye

Benefiting from the powerful capabilities of Large Language Models (LLMs), pre-trained visual encoder models connected to an LLMs can realize Vision Language Models (VLMs). However, existing research shows that the visual modality of VLMs…

Computer Vision and Pattern Recognition · Computer Science 2024-05-24 Zhendong Liu , Yuanbi Nie , Yingshui Tan , Xiangyu Yue , Qiushi Cui , Chongjun Wang , Xiaoyong Zhu , Bo Zheng

The rapid development of Multimodal Large Reasoning Models (MLRMs) has demonstrated broad application potential, yet their safety and reliability remain critical concerns that require systematic exploration. To address this gap, we conduct…

Computation and Language · Computer Science 2025-10-14 Xinyue Lou , You Li , Jinan Xu , Xiangyu Shi , Chi Chen , Kaiyu Huang

Multimodal Large Language Models (MLLMs) pose critical safety challenges, as they are susceptible not only to adversarial attacks such as jailbreaking but also to inadvertently generating harmful content for benign users. While internal…

Machine Learning · Computer Science 2026-03-17 Ming Wen , Kun Yang , Xin Chen , Jingyu Zhang , Dingding Han , Shiwen Cui , Yuedong Xu

Multimodal Large Language Models (MLLMs) are susceptible to the implicit reasoning risk, wherein innocuous unimodal inputs synergistically assemble into risky multimodal data that produce harmful outputs. We attribute this vulnerability to…

Artificial Intelligence · Computer Science 2025-09-17 Wei Cai , Shujuan Liu , Jian Zhao , Ziyan Shi , Yusheng Zhao , Yuchen Yuan , Tianle Zhang , Chi Zhang , Xuelong Li

Multimodal Large Language Models (MLLMs) are widely used in various fields due to their powerful cross-modal comprehension and generation capabilities. However, more modalities bring more vulnerabilities to being utilized for jailbreak…

Computer Vision and Pattern Recognition · Computer Science 2025-12-09 Shiji Zhao , Shukun Xiong , Yao Huang , Yan Jin , Zhenyu Wu , Jiyang Guan , Ranjie Duan , Jialing Tao , Hui Xue , Xingxing Wei

Large Vision-Language Models (LVLMs) exhibit powerful reasoning capabilities but suffer sophisticated jailbreak vulnerabilities. Fundamentally, aligning LVLMs is not just a safety challenge but a problem of economic efficiency. Current…

Artificial Intelligence · Computer Science 2026-03-17 Ruoxi Cheng , Haoxuan Ma , Teng Ma , Hongyi Zhang

Benefiting from the powerful capabilities of Large Language Models (LLMs), pre-trained visual encoder models connected to LLMs form Vision Language Models (VLMs). However, recent research shows that the visual modality in VLMs is highly…

Computer Vision and Pattern Recognition · Computer Science 2025-01-14 Zhendong Liu , Yuanbi Nie , Yingshui Tan , Jiaheng Liu , Xiangyu Yue , Qiushi Cui , Chongjun Wang , Xiaoyong Zhu , Bo Zheng

Aligning Vision-Language Models (VLMs) with safety standards is essential to mitigate risks arising from their multimodal complexity, where integrating vision and language unveils subtle threats beyond the reach of conventional safeguards.…

Machine Learning · Computer Science 2025-10-14 Menglan Chen , Xianghe Pang , Jingjing Dong , WenHao Wang , Yaxin Du , Siheng Chen

Large Multimodal Reasoning Models (LMRMs) are moving into real applications, where they must be both useful and safe. Safety is especially challenging in multimodal settings: images and text can be combined to bypass guardrails, and single…

Artificial Intelligence · Computer Science 2025-10-07 Yizhuo Ding , Mingkang Chen , Qiuhua Liu , Fenghua Weng , Wanying Qu , Yue Yang , Yugang Jiang , Zuxuan Wu , Yanwei Fu , Wenqi Shao

Large Vision-Language Models (VLMs) have achieved remarkable performance across a wide range of tasks. However, their deployment in safety-critical domains poses significant challenges. Existing safety fine-tuning methods, which focus on…

Computer Vision and Pattern Recognition · Computer Science 2026-02-04 Yi Ding , Lijun Li , Bing Cao , Jing Shao

Multimodal Large Language Models (MLLMs) are rapidly evolving, demonstrating impressive capabilities as multimodal assistants that interact with both humans and their environments. However, this increased sophistication introduces…

Artificial Intelligence · Computer Science 2025-04-24 Kaiwen Zhou , Chengzhi Liu , Xuandong Zhao , Anderson Compalas , Dawn Song , Xin Eric Wang

Medical vision-language models (VLMs) show strong performance on radiology tasks but often produce fluent yet weakly grounded conclusions due to over-reliance on a dominant modality. We introduce a context-aligned reasoning framework that…

Computer Vision and Pattern Recognition · Computer Science 2026-04-13 Sumra Khan , Sagar Chhabriya , Aizan Zafar , Sheeraz Arif , Amgad Muneer , Anas Zafar , Shaina Raza , Rizwan Qureshi

As Vision-Language Models (VLMs) move into interactive, multi-turn use, safety concerns intensify for multimodal multi-turn dialogue, which is characterized by concealment of malicious intent, contextual risk accumulation, and cross-modal…

Computer Vision and Pattern Recognition · Computer Science 2026-03-11 Guolei Huang , Qinzhi Peng , Gan Xu , Yao Huang , Yuxuan Lu , Yongjun Shen

Multimodal symbolic logical reasoning, which aims to deduce new facts from multimodal input via formal logic, is critical in high-stakes applications such as autonomous driving and medical diagnosis, as its rigorous, deterministic reasoning…

Computer Vision and Pattern Recognition · Computer Science 2026-01-30 Jundong Xu , Hao Fei , Yuhui Zhang , Liangming Pan , Qijun Huang , Qian Liu , Preslav Nakov , Min-Yen Kan , William Yang Wang , Mong-Li Lee , Wynne Hsu

Vision-Language Models (VLMs) continue to struggle to make morally salient judgments in multimodal and socially ambiguous contexts. Prior works typically rely on binary or pairwise supervision, which often fail to capture the continuous and…

Computer Vision and Pattern Recognition · Computer Science 2026-02-04 Eunkyu Park , Wesley Hanwen Deng , Cheyon Jin , Matheus Kunzler Maldaner , Jordan Wheeler , Jason I. Hong , Hong Shen , Adam Perer , Ken Holstein , Motahhare Eslami , Gunhee Kim

Recent advances in Vision-Language Models (VLMs) have improved performance in multi-modal learning, raising the question of whether these models truly understand the content they process. Crucially, can VLMs detect when a reasoning process…

Computer Vision and Pattern Recognition · Computer Science 2026-04-21 Yang Shi , Yifeng Xie , Minzhe Guo , Liangsi Lu , Mingxuan Huang , Jingchao Wang , Zhihong Zhu , Boyan Xu , Zhiqi Huang
‹ Prev 1 2 3 10 Next ›