English

Collective Kernel EFT for Pre-activation ResNets

Machine Learning 2026-04-20 v1 High Energy Physics - Theory Machine Learning

Abstract

In finite-width deep neural networks, the empirical kernel GG evolves stochastically across layers. We develop a collective kernel effective field theory (EFT) for pre-activation ResNets based on a GG-only closure hierarchy and diagnose its finite validity window. Exploiting the exact conditional Gaussianity of residual increments, we derive an exact stochastic recursion for GG. Applying Gaussian approximations systematically yields a continuous-depth ODE system for the mean kernel K0K_0, the kernel covariance V4V_4, and the 1/n1/n mean correction K1,EFTK_{1,\mathrm{EFT}}, which emerges diagrammatically as a one-loop tadpole correction. Numerically, K0K_0 remains accurate at all depths. However, the V4V_4 equation residual accumulates to an O(1)O(1) error at finite time, primarily driven by approximation errors in the GG-only transport term. Furthermore, K1,EFTK_{1,\mathrm{EFT}} fails due to the breakdown of the source closure, which exhibits a systematic mismatch even at initialization. These findings highlight the limitations of GG-only state-space reduction and suggest extending the state space to incorporate the sigma-kernel.

Cite

@article{arxiv.2604.15742,
  title  = {Collective Kernel EFT for Pre-activation ResNets},
  author = {Hidetoshi Kawase and Toshihiro Ota},
  journal= {arXiv preprint arXiv:2604.15742},
  year   = {2026}
}

Comments

20 pages

R2 v1 2026-07-01T12:13:53.067Z