English

Attributional Safety Failures in Large Language Models under Code-Mixed Perturbations

Computation and Language 2025-12-02 v2 Artificial Intelligence

Abstract

While LLMs appear robustly safety-aligned in English, we uncover a catastrophic, overlooked weakness: attributional collapse under code-mixed perturbations. Our systematic evaluation of open models shows that the linguistic camouflage of code-mixing -- ``blending languages within a single conversation'' -- can cause safety guardrails to fail dramatically. Attack success rates (ASR) spike from a benign 9\% in monolingual English to 69\% under code-mixed inputs, with rates exceeding 90\% in non-Western contexts such as Arabic and Hindi. These effects hold not only on controlled synthetic datasets but also on real-world social media traces, revealing a serious risk for billions of users. To explain why this happens, we introduce saliency drift attribution (SDA), an interpretability framework that shows how, under code-mixing, the model's internal attention drifts away from safety-critical tokens (e.g., ``violence'' or ``corruption''), effectively blinding it to harmful intent. Finally, we propose a lightweight translation-based restoration strategy that recovers roughly 80\% of the safety lost to code-mixing, offering a practical path toward more equitable and robust LLM safety.

Keywords

Cite

@article{arxiv.2505.14469,
  title  = {Attributional Safety Failures in Large Language Models under Code-Mixed Perturbations},
  author = {Somnath Banerjee and Pratyush Chatterjee and Shanu Kumar and Sayan Layek and Parag Agrawal and Rima Hazra and Animesh Mukherjee},
  journal= {arXiv preprint arXiv:2505.14469},
  year   = {2025}
}
R2 v1 2026-07-01T02:25:23.982Z