On the boundedness of the sequence generated by minibatch stochastic gradient descent
Optimization and Control
2025-07-01 v1 Functional Analysis
Abstract
Stochastic Gradient Descent (SGD) with Polyak's stepsize has recently gained renewed attention in stochastic optimization. Recently, Orvieto, Lacoste-Julien, and Loizou introduced a decreasing variant of Polyak's stepsize, where convergence relies on a boundedness assumption of the iterates. They established that this assumption holds under strong convexity. In this paper, we extend their result by proving that boundedness also holds for a broader class of objective functions, including coercive functions. We also present a case in which boundedness may or may not hold.
Cite
@article{arxiv.2506.23303,
title = {On the boundedness of the sequence generated by minibatch stochastic gradient descent},
author = {Heinz H. Bauschke and Tran Thanh Tung},
journal= {arXiv preprint arXiv:2506.23303},
year = {2025}
}