Convergence of continuous-time stochastic gradient descent with applications to deep neural networks
Machine Learning
2025-11-03 v2 Optimization and Control
Machine Learning
Abstract
We study a continuous-time approximation of the stochastic gradient descent process for minimizing the population expected loss in learning problems. The main results establish general sufficient conditions for the convergence, extending the results of Chatterjee (2022) established for (nonstochastic) gradient descent. We show how the main result can be applied to the case of overparametrized neural network training.
Keywords
Cite
@article{arxiv.2409.07401,
title = {Convergence of continuous-time stochastic gradient descent with applications to deep neural networks},
author = {Gabor Lugosi and Eulalia Nualart},
journal= {arXiv preprint arXiv:2409.07401},
year = {2025}
}