AdaShield：通过自适应盾牌提示保护多模态大型语言模型免受基于结构的攻击

随着多模态大型语言模型 (MLLMs) 的出现和广泛部署，确保其安全性变得越来越重要。然而，由于集成了额外的模态，MLLMs 面临新的漏洞，使其易受基于结构的越狱攻击的威胁，这类攻击将语义内容（例如“有害文本”）注入图像以误导 MLLMs。In this work, we aim to defend against such threats. Specifically, we propose \textbf{Ada}ptive \textbf{Shield} Prompting (\textbf{AdaShield}), which prepends inputs with defense prompts to defend MLLMs against structure-based jailbreak attacks without fine-tuning MLLMs or training additional modules (e.g., post-stage content detector). Initially, we present a manually designed static defense prompt, which thoroughly examines the image and instruction content step by step and specifies response methods to malicious queries. Furthermore, we introduce an adaptive auto-refinement framework, consisting of a target MLLM and a LLM-based defense prompt generator (Defender). These components collaboratively and iteratively communicate to generate a defense prompt. Extensive experiments on the popular structure-based jailbreak attacks and benign datasets show that our methods can consistently improve MLLMs' robustness against structure-based jailbreak attacks without compromising the model's general capabilities evaluated on standard benign tasks. Our code is available at https://github.com/rain305f/AdaShield.