Sequential convergence of AdaGrad algorithm for smooth convex optimization
Optimization and Control
2021-04-14 v3 Machine Learning
Abstract
We prove that the iterates produced by, either the scalar step size variant, or the coordinatewise variant of AdaGrad algorithm, are convergent sequences when applied to convex objective functions with Lipschitz gradient. The key insight is to remark that such AdaGrad sequences satisfy a variable metric quasi-Fej\'er monotonicity property, which allows to prove convergence.
Cite
@article{arxiv.2011.12341,
title = {Sequential convergence of AdaGrad algorithm for smooth convex optimization},
author = {Cheik Traoré and Edouard Pauwels},
journal= {arXiv preprint arXiv:2011.12341},
year = {2021}
}
Comments
9 pages