English

Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency

Computation and Language 2026-05-19 v1 Artificial Intelligence Machine Learning

Abstract

While scaling laws govern aggregate large language model performance, no scaling law has linked factual recall to both model size and training-data composition. We evaluated 38 models on over 8,900 scholarly references evaluated by an automated reference verification system. Recall quality follows a sigmoid in the log-linear combination of model parameter count and topic representation in training data. These two variables alone explain 60% of the variance across 16 dense models from four families, rising to 74-94% within individual families. The form matches a superposition-inspired account in which recall is gated by a signal-to-noise ratio: signal strength scales with concept frequency and the noise floor with model capacity.

Keywords

Cite

@article{arxiv.2605.18732,
  title  = {Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency},
  author = {Matthew L. Smith and Jonathan P. Shock and Samuel T. Segun and Iyiola E. Olatunji and Tegawendé F. Bissyandé},
  journal= {arXiv preprint arXiv:2605.18732},
  year   = {2026}
}

Comments

18 pages, 5 figures, 6 tables