English
Related papers

Related papers: Vulnerability Mitigation for Safety-Aligned Langua…

200 papers

Large Language Models (LLMs) exhibit impressive capabilities but also present risks such as biased content generation and privacy issues. One of the current alignment techniques includes principle-driven integration, but it faces challenges…

Computation and Language · Computer Science 2025-05-30 Yi Luo , Zhenghao Lin , Yuhao Zhang , Jiashuo Sun , Chen Lin , Chengjin Xu , Xiangdong Su , Yelong Shen , Jian Guo , Yeyun Gong

Vision language models (VLMs) extend the reasoning capabilities of large language models (LLMs) to cross-modal settings, yet remain highly vulnerable to multimodal jailbreak attacks. Existing defenses predominantly rely on safety…

Artificial Intelligence · Computer Science 2026-04-14 Mengxuan Wang , Yuxin Chen , Gang Xu , Tao He , Hongjie Jiang , Ming Li

Safety evaluation of multimodal foundation models often treats vision and language inputs separately, missing risks from joint interpretation where benign content becomes harmful in combination. Existing approaches also fail to distinguish…

Computer Vision and Pattern Recognition · Computer Science 2025-12-04 Shruti Palaskar , Leon Gatys , Mona Abdelrahman , Mar Jacobo , Larry Lindsey , Rutika Moharir , Gunnar Lund , Yang Xu , Navid Shiee , Jeffrey Bigham , Charles Maalouf , Joseph Yitan Cheng

Most prior safety research of large language models (LLMs) has focused on enhancing the alignment of LLMs to better suit the safety requirements of humans. However, internalizing such safeguard features into larger models brought challenges…

Computation and Language · Computer Science 2025-01-24 Ohjoon Kwon , Donghyeon Jeon , Nayoung Choi , Gyu-Hwung Cho , Changbong Kim , Hyunwoo Lee , Inho Kang , Sun Kim , Taiwoo Park

The safety mechanisms of large language models (LLMs) exhibit notable fragility, as even fine-tuning on datasets without harmful content may still undermine their safety capabilities. Meanwhile, existing safety alignment methods…

Computers and Society · Computer Science 2026-02-03 Guanghao Zhou , Panjia Qiu , Cen Chen , Hongyu Li , Mingyuan Chu , Xin Zhang , Jun Zhou

The current paradigm for safety alignment of large language models (LLMs) follows a one-size-fits-all approach: the model refuses to interact with any content deemed unsafe by the model provider. This approach lacks flexibility in the face…

Computation and Language · Computer Science 2025-03-05 Jingyu Zhang , Ahmed Elgohary , Ahmed Magooda , Daniel Khashabi , Benjamin Van Durme

Large Language Models (LLMs) have been extensively used across diverse domains, including virtual assistants, automated code generation, and scientific research. However, they remain vulnerable to jailbreak attacks, which manipulate the…

Cryptography and Security · Computer Science 2026-01-05 Haoran Gu , Handing Wang , Yi Mei , Mengjie Zhang , Yaochu Jin

Safety alignment in Large Language Models (LLMs) often involves mediating internal representations to refuse harmful requests. Recent research has demonstrated that these safety mechanisms can be bypassed by ablating or removing specific…

Computation and Language · Computer Science 2025-08-29 Harethah Abu Shairah , Hasan Abed Al Kader Hammoud , George Turkiyyah , Bernard Ghanem

Large Language Models (LLMs) have emerged as promising solutions for a variety of medical and clinical decision support applications. However, LLMs are often subject to different types of biases, which can lead to unfair treatment of…

Computation and Language · Computer Science 2024-08-23 Raphael Poulain , Hamed Fayyaz , Rahmatollah Beheshti

Vision Language Models (VLMs) have become essential backbones for multimodal intelligence, yet significant safety challenges limit their real-world application. While textual inputs are often effectively safeguarded, adversarial visual…

Computer Vision and Pattern Recognition · Computer Science 2025-02-11 Yi Ding , Bolian Li , Ruqi Zhang

Diffusion language models (DLMs) generate tokens in parallel through iterative denoising, which can reduce latency and enable bidirectional conditioning. However, the safety risks posed by jailbreak attacks that exploit this inference…

Artificial Intelligence · Computer Science 2026-02-18 Shojiro Yamabe , Jun Sakuma

Alignment of large language models remains a central challenge in natural language processing. Preference optimization has emerged as a popular and effective method for improving alignment, typically through training-time or prompt-based…

Machine Learning · Computer Science 2025-10-01 Frédéric Berdoz , Luca A. Lanzendörfer , René Caky , Roger Wattenhofer

The problem of algorithmic bias in machine learning has gained a lot of attention in recent years due to its concrete and potentially hazardous implications in society. In much the same manner, biases can also alter modern industrial and…

Machine Learning · Computer Science 2022-10-11 Laurent Risser , Agustin Picard , Lucas Hervier , Jean-Michel Loubes

Vulnerability of Frontier language models to misuse and jailbreaks has prompted the development of safety measures like filters and alignment training in an effort to ensure safety through robustness to adversarially crafted prompts. We…

Cryptography and Security · Computer Science 2024-10-31 David Glukhov , Ziwen Han , Ilia Shumailov , Vardan Papyan , Nicolas Papernot

A key concern for AI safety remains understudied in the machine learning (ML) literature: how can we ensure users of ML models do not leverage predictions on incorrect personal data to harm others? This is particularly pertinent given the…

Machine Learning · Computer Science 2025-10-01 Muhammad H. Ashiq , Peter Triantafillou , Hung Yun Tseng , Grigoris G. Chrysos

The rapid advancement of ML models in critical sectors such as healthcare, finance, and security has intensified the need for robust data security, model integrity, and reliable outputs. Large multimodal foundational models, while crucial…

Cryptography and Security · Computer Science 2024-12-13 Hongyang Zhang , Yue Zhao , Claudio Angione , Harry Yang , James Buban , Ahmad Farhan , Fielding Johnston , Patrick Colangelo

Vision-language-action models (VLAs) show potential as generalist robot policies. However, these models pose extreme safety challenges during real-world deployment, including the risk of harm to the environment, the robot itself, and…

Robotics · Computer Science 2026-04-21 Borong Zhang , Yuhao Zhang , Jiaming Ji , Yingshan Lei , Yishuai Cai , Josef Dai , Yuanpei Chen , Yaodong Yang

With the growing deployment of Vision-Language Models (VLMs) in real-world applications, previously overlooked safety risks are becoming increasingly evident. In particular, seemingly innocuous multimodal inputs can combine to reveal…

Computer Vision and Pattern Recognition · Computer Science 2025-10-07 Youngjin Na , Sangheon Jeong , Youngwan Lee , Jian Lee , Dawoon Jeong , Youngman Kim

Safety post-training can improve the harmfulness and policy compliance of Large Language Models (LLMs), but it may also reduce general utility, a phenomenon often described as the \emph{alignment tax}. We study this trade-off through the…

Machine Learning · Computer Science 2026-05-13 Guanglong Sun , Siyuan Zhang , Liyuan Wang , Jun Zhu , Hang Su , Yi Zhong

Fine-tuning large language models (LLMs) on additional datasets is often necessary to optimize them for specific downstream tasks. However, existing safety alignment measures, which restrict harmful behavior during inference, are…

Computation and Language · Computer Science 2024-10-15 Minjun Zhu , Linyi Yang , Yifan Wei , Ningyu Zhang , Yue Zhang