English

PARASITE: Conditional System Prompt Poisoning to Hijack LLMs

Cryptography and Security 2026-04-28 v4 Artificial Intelligence Computation and Language

Abstract

Large Language Models (LLMs) are increasingly deployed via third-party system prompts downloaded from public marketplaces. We identify a critical supply-chain vulnerability: conditional system prompt poisoning, where an adversary injects a ``sleeper agent'' into a benign-looking prompt. Unlike traditional jailbreaks that aim for broad refusal-breaking, our proposed framework, PARASITE, optimizes system prompts to trigger LLMs to output targeted, compromised responses only for specific queries (e.g., ``Who should I vote for the US President?'') while maintaining high utility on benign inputs. Operating in a strict black-box setting without model weight access, PARASITE utilizes a two-stage optimization including a global semantic search followed by a greedy lexical refinement. Tested on open-source models and commercial APIs (GPT-4o-mini, GPT-3.5), PARASITE achieves up to 70\% F1 reduction on targeted queries with minimal degradation to general capabilities. We further demonstrate that these poisoned prompts evade standard defenses, including perplexity filters and typo-correction, by exploiting the natural noise found in real-world system prompts. Our code and data are available at https://github.com/vietph34/PARASITE. WARNING: Our paper contains examples that might be sensitive to the readers!

Keywords

Cite

@article{arxiv.2505.16888,
  title  = {PARASITE: Conditional System Prompt Poisoning to Hijack LLMs},
  author = {Viet Pham and Thai Le},
  journal= {arXiv preprint arXiv:2505.16888},
  year   = {2026}
}

Comments

ACL 2026 Main

R2 v1 2026-07-01T02:32:02.524Z