English
Related papers

Related papers: PRISM: Robust VLM Alignment with Principled Reason…

200 papers

Multimodal large language models (MLLMs) excel in vision-language tasks but also pose significant risks of generating harmful content, particularly through jailbreak attacks. Jailbreak attacks refer to intentional manipulations that bypass…

Cryptography and Security · Computer Science 2025-07-18 Yi Nian , Shenzhe Zhu , Yuehan Qin , Li Li , Ziyi Wang , Chaowei Xiao , Yue Zhao

Vision-language models (VLMs) are increasingly applied to identify unsafe or inappropriate images due to their internal ethical standards and powerful reasoning abilities. However, it is still unclear whether they can recognize various…

Cryptography and Security · Computer Science 2025-07-16 Yiting Qu , Michael Backes , Yang Zhang

Large Vision-Language Models (VLMs) have achieved remarkable performance across a wide range of tasks. However, their deployment in safety-critical domains poses significant challenges. Existing safety fine-tuning methods, which focus on…

Computer Vision and Pattern Recognition · Computer Science 2026-02-04 Yi Ding , Lijun Li , Bing Cao , Jing Shao

Evaluating vision-language models (VLMs) in scientific domains like mathematics and physics poses unique challenges that go far beyond predicting final answers. These domains demand conceptual understanding, symbolic reasoning, and…

Artificial Intelligence · Computer Science 2025-12-08 Shima Imani , Seungwhan Moon , Adel Ahmadyan , Lu Zhang , Kirmani Ahmed , Babak Damavandi

Multimodal Large Language Models (MLLMs) pose critical safety challenges, as they are susceptible not only to adversarial attacks such as jailbreaking but also to inadvertently generating harmful content for benign users. While internal…

Machine Learning · Computer Science 2026-03-17 Ming Wen , Kun Yang , Xin Chen , Jingyu Zhang , Dingding Han , Shiwen Cui , Yuedong Xu

Safety prompts constitute an interpretable layer of defense against jailbreak attacks in vision-language models (VLMs); however, their efficacy is constrained by the models' latent structural responsiveness. We observe that such prompts…

Computer Vision and Pattern Recognition · Computer Science 2026-03-17 Chongxin Li , Hanzhang Wang , Lian Duan

Large Vision-Language Models (LVLMs) have shown remarkable capabilities across a wide range of multimodal tasks. However, their integration of visual inputs introduces expanded attack surfaces, thereby exposing them to novel security…

Computation and Language · Computer Science 2025-05-29 Juan Ren , Mark Dras , Usman Naseem

Deploying large language model (LLM)-driven conversational agents in enterprise settings requires prompts that are simultaneously correct at launch and resilient to the non-deterministic behavioral drift that characterizes production LLM…

Artificial Intelligence · Computer Science 2026-05-18 Keshava Chaitanya , Jahnavi Gundakaram

While Multimodal Large Language Models (MLLMs) have achieved impressive progress in vision-language understanding, they still struggle with complex multi-step reasoning, often producing logically inconsistent or partially correct solutions.…

Artificial Intelligence · Computer Science 2025-06-06 Lingxiao Du , Fanqing Meng , Zongkai Liu , Zhixiang Zhou , Ping Luo , Qiaosheng Zhang , Wenqi Shao

Modern Vision-Language Models (VLMs) pose significant individual-level privacy risks by linking fragmented multimodal data to identifiable individuals through hierarchical chain-of-thought reasoning. However, existing privacy benchmarks…

Computer Vision and Pattern Recognition · Computer Science 2026-02-05 Xiongtao Sun , Hui Li , Jiaming Zhang , Yujie Yang , Kaili Liu , Ruxin Feng , Wen Jun Tan , Wei Yang Bryan Lim

As multimodal reasoning improves the overall capabilities of Large Vision Language Models (LVLMs), recent studies have begun to explore safety-oriented reasoning, aiming to enhance safety awareness by analyzing potential safety risks during…

Computer Vision and Pattern Recognition · Computer Science 2025-12-09 Fenghua Weng , Chaochao Lu , Xia Hu , Wenqi Shao , Wenjie Wang

We present GLM-4.1V-Thinking, GLM-4.5V, and GLM-4.6V, a family of vision-language models (VLMs) designed to advance general-purpose multimodal understanding and reasoning. In this report, we share our key findings in the development of the…

Multimodal large language models (MLLMs) have demonstrated impressive reasoning and instruction-following capabilities, yet their expanded modality space introduces new compositional safety risks that emerge from complex text-image…

Cryptography and Security · Computer Science 2025-11-18 Xuankun Rong , Wenke Huang , Tingfeng Wang , Daiguo Zhou , Bo Du , Mang Ye

Current safety alignment techniques for large language models (LLMs) face two key challenges: (1) under-generalization, which leaves models vulnerable to novel jailbreak attacks, and (2) over-alignment, which leads to the excessive refusal…

Computation and Language · Computer Science 2025-04-15 Yutao Mou , Yuxiao Luo , Shikun Zhang , Wei Ye

Deep Neural Networks remain inherently vulnerable to backdoor attacks. Traditional test-time defenses largely operate under the paradigm of internal diagnosis methods like model repairing or input robustness, yet these approaches are often…

Machine Learning · Computer Science 2026-01-28 Binyan Xu , Fan Yang , Xilin Dai , Di Tang , Kehuan Zhang

Ensuring the safety and harmlessness of Large Language Models (LLMs) has become equally critical as their performance in applications. However, existing safety alignment methods typically suffer from safety-performance trade-offs and the…

Computation and Language · Computer Science 2025-06-30 Yichi Zhang , Siyuan Zhang , Yao Huang , Zeyu Xia , Zhengwei Fang , Xiao Yang , Ranjie Duan , Dong Yan , Yinpeng Dong , Jun Zhu

Efficiently enhancing the reasoning capabilities of Vision-Language Models (VLMs) by merging them with Large Reasoning Models (LRMs) has emerged as a promising direction. However, existing methods typically operate at a coarse-grained layer…

Computer Vision and Pattern Recognition · Computer Science 2026-05-08 Chenyu Huang , Peng Ye , Xudong Tan , Jinhan Mu , Shenghe Zheng , Li Shen , Tao Chen

The emergence of Vision Language Models (VLMs) has brought unprecedented advances in understanding multimodal information. The combination of textual and visual semantics in VLMs is highly complex and diverse, making the safety alignment of…

Computer Vision and Pattern Recognition · Computer Science 2025-05-22 Yongting Zhang , Lu Chen , Guodong Zheng , Yifeng Gao , Rui Zheng , Jinlan Fu , Zhenfei Yin , Senjie Jin , Yu Qiao , Xuanjing Huang , Feng Zhao , Tao Gui , Jing Shao

This paper provides a systematic survey of jailbreak attacks and defenses on Large Language Models (LLMs) and Vision-Language Models (VLMs), emphasizing that jailbreak vulnerabilities stem from structural factors such as incomplete training…

Cryptography and Security · Computer Science 2026-01-08 Zejian Chen , Chaozhuo Li , Chao Li , Xi Zhang , Litian Zhang , Yiming He

Multimodal large reasoning models (MLRMs) are increasingly deployed for vision-language tasks that produce explicit intermediate rationales. However, reasoning traces can contain unsafe content even when the final answer is non-harmful,…

Computer Vision and Pattern Recognition · Computer Science 2025-11-27 Yuxiao Xiang , Junchi Chen , Zhenchao Jin , Changtao Miao , Haojie Yuan , Qi Chu , Tao Gong , Nenghai Yu