English

From Prerequisites to Predictions: Validating a Geometric Hallucination Taxonomy Through Controlled Induction

Computation and Language 2026-03-03 v1

Abstract

We test whether a geometric hallucination taxonomy -- classifying failures as center-drift (Type~1), wrong-well convergence (Type~2), or coverage gaps (Type~3) -- can distinguish hallucination types through controlled induction in GPT-2. Using a two-level statistical design with prompts (N=15N = 15/group) as the unit of inference, we run each experiment 20 times with different generation seeds to quantify result stability. In static embeddings, Type~3 norm separation is robust (significant in 18/20 runs, Holm-corrected in 14/20, median r=+0.61r = +0.61). In contextual hidden states, the Type~3 norm effect direction is stable (19/20 runs) but underpowered at N=15N = 15 (significant in 4/20, median r=0.28r = -0.28). Types~1 and~2 do not separate in either space (3/20{\leq}\,3/20 runs). Token-level tests inflate significance by 4--16×\times through pseudoreplication -- a finding replicated across all 20 runs. The results establish coverage-gap hallucinations as the most geometrically distinctive failure mode, carried by magnitude rather than direction, and confirm the Type~1/2 non-separation as genuine at 124M parameters.

Cite

@article{arxiv.2603.00307,
  title  = {From Prerequisites to Predictions: Validating a Geometric Hallucination Taxonomy Through Controlled Induction},
  author = {Matic Korun},
  journal= {arXiv preprint arXiv:2603.00307},
  year   = {2026}
}

Comments

9 pages, 2 figures, appendices (reproducibility, sample generation, additional figures)

R2 v1 2026-07-01T10:56:37.281Z