English

Grokking as a Falsifiable Finite-Size Transition

Machine Learning 2026-03-27 v1 Statistical Mechanics Artificial Intelligence

Abstract

Grokking -- the delayed onset of generalization after early memorization -- is often described with phase-transition language, but that claim has lacked falsifiable finite-size inputs. Here we supply those inputs by treating the group order pp of Zp\mathbb{Z}_p as an admissible extensive variable and a held-out spectral head-tail contrast as a representation-level order parameter, then apply a condensed-matter-style diagnostic chain to coarse-grid sweeps and a dense near-critical addition audit. Binder-like crossings reveal a shared finite-size boundary, and susceptibility comparison strongly disfavors a smooth-crossover interpretation (ΔAIC=16.8\Delta\mathrm{AIC}=16.8 in the near-critical audit). Phase-transition language in grokking can therefore be tested as a quantitative finite-size claim rather than invoked as analogy alone, although the transition order remains unresolved at present.

Keywords

Cite

@article{arxiv.2603.24746,
  title  = {Grokking as a Falsifiable Finite-Size Transition},
  author = {Yuda Bi and Chenyu Zhang and Qiheng Wang and Vince D Calhoun},
  journal= {arXiv preprint arXiv:2603.24746},
  year   = {2026}
}