Stochastic Adaptive Gradient Descent Without Descent
Machine Learning
2025-09-19 v1 Optimization and Control
Machine Learning
Abstract
We introduce a new adaptive step-size strategy for convex optimization with stochastic gradient that exploits the local geometry of the objective function only by means of a first-order stochastic oracle and without any hyper-parameter tuning. The method comes from a theoretically-grounded adaptation of the Adaptive Gradient Descent Without Descent method to the stochastic setting. We prove the convergence of stochastic gradient descent with our step-size under various assumptions, and we show that it empirically competes against tuned baselines.
Cite
@article{arxiv.2509.14969,
title = {Stochastic Adaptive Gradient Descent Without Descent},
author = {Jean-François Aujol and Jérémie Bigot and Camille Castera},
journal= {arXiv preprint arXiv:2509.14969},
year = {2025}
}