Improved Learning Rates for Stochastic Optimization
Abstract
Stochastic optimization is a cornerstone of modern machine learning. This paper studies the generalization performance of two classical stochastic optimization algorithms: stochastic gradient descent (SGD) and Nesterov's accelerated gradient (NAG). We establish new learning rates for both algorithms, with improved guarantees in some settings or comparable rates under weaker assumptions in others. We also provide numerical experiments to support the theory.
Cite
@article{arxiv.2107.08686,
title = {Improved Learning Rates for Stochastic Optimization},
author = {Shaojie Li and Pengwei Tang and Yong Liu},
journal= {arXiv preprint arXiv:2107.08686},
year = {2026}
}
Comments
This version substantially revises and supersedes all previous versions. Earlier versions contained errors and should not be relied upon for the current results or statements. The manuscript has been thoroughly rewritten, with a narrowed scope, a simplified presentation, a revised focus, and corresponding updates to the title and main claims. Please refer to and cite the current version