English

Breaking the Illusion of Identity in LLM Tooling

Software Engineering 2026-04-10 v1 Artificial Intelligence

Abstract

Large language models (LLMs) in research and development toolchains produce output that triggers attribution of agency and understanding -- a cognitive illusion that degrades verification behavior and trust calibration. No existing mitigation provides a systematic, deployable constraint set for output register. This paper proposes seven output-side rules, each targeting a documented linguistic mechanism, and validates them empirically. In 780 two-turn conversations (constrained vs. default register, 30 tasks, 13 replicates, 1560 API calls), anthropomorphic markers dropped from 1233 to 33 (>97% reduction, p < 0.001), outputs were 49% shorter by word count, and adapted AnthroScore confirmed the shift toward machine register (-1.94 vs. -0.96, p < 0.001). The rules are implemented as a configuration-file system prompt requiring no model modification; validation uses a single model (Claude Sonnet 4). Output quality under the constrained register was not evaluated. The mechanism is extensible to other domains.

Keywords

Cite

@article{arxiv.2604.07398,
  title  = {Breaking the Illusion of Identity in LLM Tooling},
  author = {Marek Miller},
  journal= {arXiv preprint arXiv:2604.07398},
  year   = {2026}
}

Comments

8 pages, 2 figures, 2 tables