English
Related papers

Related papers: MSTS: A Multimodal Safety Test Suite for Vision-La…

200 papers

Moral reasoning is fundamental to safe Artificial Intelligence (AI), yet ensuring its consistency across modalities becomes critical as AI systems evolve from text-based assistants to embodied agents. Current safety techniques demonstrate…

Artificial Intelligence · Computer Science 2026-03-18 Xinyi Yang , Chenheng Xu , Weijun Hong , Ce Mo , Qian Wang , Fang Fang , Yixin Zhu

Large Vision-Language Models (VLMs) have achieved remarkable performance across a wide range of tasks. However, their deployment in safety-critical domains poses significant challenges. Existing safety fine-tuning methods, which focus on…

Computer Vision and Pattern Recognition · Computer Science 2026-02-04 Yi Ding , Lijun Li , Bing Cao , Jing Shao

Multimodal large language models (MLLMs) are increasingly deployed as assistants that interact through text and images, making it crucial to evaluate contextual safety when risk depends on both the visual scene and the evolving dialogue.…

Computation and Language · Computer Science 2026-01-13 Zheyuan Liu , Dongwhi Kim , Yixin Wan , Xiangchi Yuan , Zhaoxuan Tan , Fengran Mo , Meng Jiang

The deployment of multimodal large language models (MLLMs) has brought forth a unique vulnerability: susceptibility to malicious attacks through visual inputs. This paper investigates the novel challenge of defending MLLMs against such…

Cryptography and Security · Computer Science 2024-06-18 Renjie Pi , Tianyang Han , Jianshu Zhang , Yueqi Xie , Rui Pan , Qing Lian , Hanze Dong , Jipeng Zhang , Tong Zhang

Rapid deployment of vision-language models (VLMs) magnifies safety risks, yet most evaluations rely on artificial images. This study asks: How safe are current VLMs when confronted with meme images that ordinary users share? To investigate…

Computation and Language · Computer Science 2025-09-24 DongGeon Lee , Joonwon Jang , Jihae Jeong , Hwanjo Yu

The robust safety of Vision-Language Large Models (VLLMs) against joint multilingual and multimodal threats remains severely underexplored. Current benchmarks typically isolate these dimensions, being either multilingual but text-only, or…

Computer Vision and Pattern Recognition · Computer Science 2026-04-22 Enyi Shi , Pengyang Shao , Yanxin Zhang , Chenhang Cui , Jiayi Lyu , Xiaobo Xia , Fei Shen , Tat-Seng Chua

As Multimodal Large Language Models (MLLMs) become an indispensable assistant in human life, the unsafe content generated by MLLMs poses a danger to human behavior, perpetually overhanging human society like a sword of Damocles. To…

Computation and Language · Computer Science 2026-04-21 Xinyue Lou , Jinan Xu , Jingyi Yin , Xiaolong Wang , Zhaolu Kang , Youwei Liao , Yixuan Wang , Xiangyu Shi , Fengran Mo , Su Yao , Kaiyu Huang

Building safe Large Language Models (LLMs) across multiple languages is essential in ensuring both safe access and linguistic diversity. To this end, we conduct a large-scale, comprehensive safety evaluation of the current LLM landscape.…

Computation and Language · Computer Science 2025-06-24 Felix Friedrich , Simone Tedeschi , Patrick Schramowski , Manuel Brack , Roberto Navigli , Huu Nguyen , Bo Li , Kristian Kersting

Large language models (LLMs) and small language models (SLMs) are being adopted at remarkable speed, although their safety still remains a serious concern. With the advent of multilingual S/LLMs, the question now becomes a matter of scale:…

Large visual-language models (LVLMs) integrate aligned large language models (LLMs) with visual modules to process multimodal inputs. However, the safety mechanisms developed for text-based LLMs do not naturally extend to visual modalities,…

Computer Vision and Pattern Recognition · Computer Science 2025-07-29 Shen Li , Liuyi Yao , Wujia Niu , Lan Zhang , Yaliang Li

Safeguarding vision-language models (VLMs) is a critical challenge, as existing methods often suffer from over-defense, which harms utility, or rely on shallow alignment, failing to detect complex threats that require deep reasoning. To…

Cryptography and Security · Computer Science 2026-04-03 Nanxi Li , Zhengyue Zhao , G. Edward Suh , Marco Pavone , Chaowei Xiao

Multimodal large language models (MLLMs) enable interaction over both text and images, but their safety behavior can be driven by unimodal shortcuts instead of true joint intent understanding. We introduce CSR-Bench, a benchmark for…

Artificial Intelligence · Computer Science 2026-02-04 Yuxuan Liu , Yuntian Shi , Kun Wang , Haoting Shen , Kun Yang

As Vision-Language Models (VLMs) move into interactive, multi-turn use, safety concerns intensify for multimodal multi-turn dialogue, which is characterized by concealment of malicious intent, contextual risk accumulation, and cross-modal…

Computer Vision and Pattern Recognition · Computer Science 2026-03-11 Guolei Huang , Qinzhi Peng , Gan Xu , Yao Huang , Yuxuan Lu , Yongjun Shen

Safety lies at the core of developing and deploying large language models (LLMs). However, previous safety benchmarks only concern the safety in one language, e.g. the majority language in the pretraining data such as English. In this work,…

Computation and Language · Computer Science 2024-06-21 Wenxuan Wang , Zhaopeng Tu , Chang Chen , Youliang Yuan , Jen-tse Huang , Wenxiang Jiao , Michael R. Lyu

Large-scale Vision Language Models (LVLMs) exhibit advanced capabilities in tasks that require visual information, including object detection. These capabilities have promising applications in various industrial domains, such as autonomous…

Computer Vision and Pattern Recognition · Computer Science 2025-12-01 Haruki Sakajo , Hiroshi Takato , Hiroshi Tsutsui , Komei Soda , Hidetaka Kamigaito , Taro Watanabe

Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal perception and generation, yet their safety alignment remains a critical challenge.Existing defenses and vulnerable to multimodal jailbreaks, as visual…

Artificial Intelligence · Computer Science 2025-10-21 MingSheng Li , Guangze Zhao , Sichen Liu

Despite emerging efforts to enhance the safety of Vision-Language Models (VLMs), current approaches face two main shortcomings. 1) Existing safety-tuning datasets and benchmarks only partially consider how image-text interactions can yield…

Computer Vision and Pattern Recognition · Computer Science 2025-11-26 Youngwan Lee , Kangsan Kim , Kwanyong Park , Ilcahe Jung , Soojin Jang , Seanie Lee , Yong-Ju Lee , Sung Ju Hwang

Artificial Intelligence (AI) is revolutionizing scientific research, yet its growing integration into laboratory environments presents critical safety challenges. Large language models (LLMs) and vision language models (VLMs) now assist in…

As large language models (LLMs) rapidly evolve, they bring significant conveniences to our work and daily lives, but also introduce considerable safety risks. These models can generate texts with social biases or unethical content, and…

Computation and Language · Computer Science 2024-10-30 Zhihao Liu , Chenhui Hu

The emergence of Vision Language Models (VLMs) has brought unprecedented advances in understanding multimodal information. The combination of textual and visual semantics in VLMs is highly complex and diverse, making the safety alignment of…

Computer Vision and Pattern Recognition · Computer Science 2025-05-22 Yongting Zhang , Lu Chen , Guodong Zheng , Yifeng Gao , Rui Zheng , Jinlan Fu , Zhenfei Yin , Senjie Jin , Yu Qiao , Xuanjing Huang , Feng Zhao , Tao Gui , Jing Shao