Mitigating Trojanized Prompt Chains in Educational LLM Use Cases: Experimental Findings and Detection Tool Design

Richard M. Charles; James H. Curry; Richard B. Charles

Mitigating Trojanized Prompt Chains in Educational LLM Use Cases: Experimental Findings and Detection Tool Design

Cryptography and Security 2025-07-22 v1 Artificial Intelligence

Authors: Richard M. Charles , James H. Curry , Richard B. Charles

Abstract

The integration of Large Language Models (LLMs) in K--12 education offers both transformative opportunities and emerging risks. This study explores how students may Trojanize prompts to elicit unsafe or unintended outputs from LLMs, bypassing established content moderation systems with safety guardrils. Through a systematic experiment involving simulated K--12 queries and multi-turn dialogues, we expose key vulnerabilities in GPT-3.5 and GPT-4. This paper presents our experimental design, detailed findings, and a prototype tool, TrojanPromptGuard (TPG), to automatically detect and mitigate Trojanized educational prompts. These insights aim to inform both AI safety researchers and educational technologists on the safe deployment of LLMs for educators.

Keywords

prompt injection LLM security Trojan attack educational technology content moderation AI safety

Cite

@article{arxiv.2507.14207,
  title  = {Mitigating Trojanized Prompt Chains in Educational LLM Use Cases: Experimental Findings and Detection Tool Design},
  author = {Richard M. Charles and James H. Curry and Richard B. Charles},
  journal= {arXiv preprint arXiv:2507.14207},
  year   = {2025}
}

Comments

12 pages, 1 figure

Mitigating Trojanized Prompt Chains in Educational LLM Use Cases: Experimental Findings and Detection Tool Design

Abstract

Keywords

Cite

Comments

Related papers