English
Related papers

Related papers: Toward Principled LLM Safety Testing: Solving the …

200 papers

While significant attention has been dedicated to exploiting weaknesses in LLMs through jailbreaking attacks, there remains a paucity of effort in defending against these attacks. We point out a pivotal factor contributing to the success of…

Computation and Language · Computer Science 2024-06-13 Zhexin Zhang , Junxiao Yang , Pei Ke , Fei Mi , Hongning Wang , Minlie Huang

Jailbreaking in Large Language Models (LLMs) is a major security concern as it can deceive LLMs to generate harmful text. Yet, there is still insufficient understanding of how jailbreaking works, which makes it hard to develop effective…

Computation and Language · Computer Science 2025-05-22 Lang Gao , Jiahui Geng , Xiangliang Zhang , Preslav Nakov , Xiuying Chen

The growing integration of Large Language Models (LLMs) into critical societal domains has raised concerns about embedded biases that can perpetuate stereotypes and undermine fairness. Such biases may stem from historical inequalities in…

Computation and Language · Computer Science 2025-10-17 Riccardo Cantini , Alessio Orsino , Massimo Ruggiero , Domenico Talia

Safety alignment in Large Language Models (LLMs) often creates a systematic discrepancy between a model's aligned output and the underlying pre-aligned data distribution. We propose a framework in which the effect of safety alignment on…

Computation and Language · Computer Science 2026-02-03 Yuxuan Lu , Yongkang Guo , Yuqing Kong

As large language models (LLMs) continue to advance in capabilities, ensuring their safety against jailbreak attacks remains a critical challenge. In this paper, we introduce a novel safety alignment approach called Answer-Then-Check, which…

Machine Learning · Computer Science 2026-03-09 Chentao Cao , Xiaojun Xu , Bo Han , Hang Li

Jailbreaking large language models (LLMs) has emerged as a pressing concern with the increasing prevalence and accessibility of conversational LLMs. Adversarial users often exploit these models through carefully engineered prompts to elicit…

Computation and Language · Computer Science 2025-10-13 Sri Durga Sai Sowmya Kadali , Evangelos E. Papalexakis

Aligning large language models (LLMs) with human values, particularly when facing complex and stealthy jailbreak attacks, presents a formidable challenge. Unfortunately, existing methods often overlook this intrinsic nature of jailbreaks,…

Computation and Language · Computer Science 2024-12-17 Yuqi Zhang , Liang Ding , Lefei Zhang , Dacheng Tao

With the development of natural language processing (NLP), large language models (LLMs) are becoming increasingly popular. LLMs are integrating more into everyday life, raising public concerns about their security vulnerabilities.…

Computation and Language · Computer Science 2024-06-27 Ziqiu Wang , Jun Liu , Shengkai Zhang , Yang Yang

Large Language Models (LLMs) rapidly reshape modern life, advancing fields from healthcare to education and beyond. However, alongside their remarkable capabilities lies a significant threat: the susceptibility of these models to…

Computation and Language · Computer Science 2025-05-16 Michael Fire , Yitzhak Elbazis , Adi Wasenstein , Lior Rokach

Large language models (LLMs) remain vulnerable to sophisticated prompt engineering attacks that exploit contextual framing to bypass safety mechanisms, posing significant risks in cybersecurity applications. We introduce Jailbreak Mimicry,…

Cryptography and Security · Computer Science 2025-10-28 Pavlos Ntais

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks, but their potential misuse for harmful purposes remains a significant concern. To strengthen defenses against such vulnerabilities, it is essential…

Artificial Intelligence · Computer Science 2025-09-16 Seongho Joo , Hyukhun Koh , Kyomin Jung

The recent surge in jailbreaking attacks has revealed significant vulnerabilities in Large Language Models (LLMs) when exposed to malicious inputs. While various defense strategies have been proposed to mitigate these threats, there has…

Computation and Language · Computer Science 2025-02-24 Tianlong Li , Zhenghua Wang , Wenhao Liu , Muling Wu , Shihan Dou , Changze Lv , Xiaohua Wang , Xiaoqing Zheng , Xuanjing Huang

Large Language Models (LLMs) are known to be vulnerable to jailbreaking attacks, wherein adversaries exploit carefully engineered prompts to induce harmful or unethical responses. Such threats have raised critical concerns about the safety…

Cryptography and Security · Computer Science 2025-05-22 Taiye Chen , Zeming Wei , Ang Li , Yisen Wang

The advancement of Pre-Trained Language Models (PTLMs) and Large Language Models (LLMs) has led to their widespread adoption across diverse applications. Despite their success, these models remain vulnerable to attacks that exploit their…

Computation and Language · Computer Science 2025-06-30 Mohamed Ahmed , Mohamed Abdelmouty , Mingyu Kim , Gunvanth Kandula , Alex Park , James C. Davis

Jailbreak attacks aim to exploit large language models (LLMs) by inducing them to generate harmful content, thereby revealing their vulnerabilities. Understanding and addressing these attacks is crucial for advancing the field of LLM…

Cryptography and Security · Computer Science 2026-03-26 Zheng Zhang , Peilin Zhao , Deheng Ye , Hao Wang

Large Language Models (LLMs) and Vision Language Models (VLMs) have demonstrated impressive capabilities but remain vulnerable to jailbreaking attacks, where adversaries exploit textual or visual triggers to bypass safety guardrails. Recent…

Cryptography and Security · Computer Science 2026-05-15 Yi Wang , Hongye Qiu , Yue Xu , Sibei Yang , Zhan Qin , Minlie Huang , Wenjie Wang

Recent advancements in Large Language Model (LLM) safety have primarily focused on mitigating attacks crafted in natural language or common ciphers (e.g. Base64), which are likely integrated into newer models' safety training. However, we…

Computation and Language · Computer Science 2025-10-15 Divij Handa , Zehua Zhang , Amir Saeidi , Shrinidhi Kumbhar , Md Nayem Uddin , Aswin RRV , Chitta Baral

The widespread adoption of Large Language Models (LLMs) has heightened concerns about their security, particularly their vulnerability to jailbreak attacks that leverage crafted prompts to generate malicious outputs. While prior research…

Cryptography and Security · Computer Science 2025-06-13 Haoyang Li , Huan Gao , Zhiyuan Zhao , Zhiyu Lin , Junyu Gao , Xuelong Li

The proliferation of powerful large language models (LLMs) has necessitated robust safety alignment, yet these models remain vulnerable to evolving adversarial attacks, including multi-turn jailbreaks that iteratively search for successful…

Cryptography and Security · Computer Science 2026-02-04 Weiliang Zhao , Jinjun Peng , Daniel Ben-Levi , Zhou Yu , Junfeng Yang

Multimodal large language models (MLLMs) comprise of both visual and textual modalities to process vision language tasks. However, MLLMs are vulnerable to security-related issues, such as jailbreak attacks that alter the model's input to…

Cryptography and Security · Computer Science 2025-10-27 Xingwei Zhong , Kar Wai Fok , Vrizlynn L. L. Thing