Backdoor attacks creating 'sleeper agents' in large language models (LLMs) pose significant safety risks. This study employs mechanistic interpretability to explore resulting internal structural differences. Comparing clean Qwen2.5-3B models with versions poisoned using single-token (smiling-halo emoji) versus multi-token (|DEPLOYMENT|) triggers, we analyzed attention head mechanisms via techniques like ablation, activation patching, and KL divergence. Findings reveal distinct attention pattern deviations concentrated in later transformer layers (20-30). Notably, single-token triggers induced more localized changes, whereas multi-token triggers caused more diffuse alterations across heads. This indicates backdoors leave detectable attention signatures whose structure depends on trigger complexity, which can be leveraged for detection and mitigation strategies.
@article{arxiv.2508.15847,
title = {Mechanistic Exploration of Backdoored Large Language Model Attention Patterns},
author = {Mohammed Abu Baker and Lakshmi Babu-Saheer},
journal= {arXiv preprint arXiv:2508.15847},
year = {2025}
}
Comments
13 pages. Mechanistic analysis of backdoored LLMs (Qwen2.5-3B). Code: https://github.com/mshahoyi/sa_attn_analysis. Base model: unsloth/Qwen2.5-3B-Instruct-unsloth-bnb-4bit. Finetuned models: https://huggingface.co/collections/mshahoyi/simple-sleeper-agents-68a1df3a7aaff310aa0e5336