Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
Disordered Systems and Neural Networks
2026-05-08 v3 Machine Learning
Machine Learning
Abstract
We derive a novel deterministic equivalence for the two-point function of a random matrix resolvent. Using this result, we give a unified derivation of the performance of a wide variety of high-dimensional linear models trained with stochastic gradient descent. This includes high-dimensional linear regression, kernel regression, and linear random feature models. Our results include previously known asymptotics as well as novel ones.
Cite
@article{arxiv.2502.05074,
title = {Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models},
author = {Alexander Atanasov and Blake Bordelon and Jacob A. Zavatone-Veth and Courtney Paquette and Cengiz Pehlevan},
journal= {arXiv preprint arXiv:2502.05074},
year = {2026}
}
Comments
22 pages, in press at Advances in Theoretical and Mathematical Physics