Parameter-free version of Adaptive Gradient Methods for Strongly-Convex Functions
Machine Learning
2023-07-18 v2 Optimization and Control
Abstract
The optimal learning rate for adaptive gradient methods applied to {\lambda}-strongly convex functions relies on the parameters {\lambda} and learning rate {\eta}. In this paper, we adapt a universal algorithm along the lines of Metagrad, to get rid of this dependence on {\lambda} and {\eta}. The main idea is to concurrently run multiple experts and combine their predictions to a master algorithm. This master enjoys O(d log T) regret bounds.
Cite
@article{arxiv.2306.06613,
title = {Parameter-free version of Adaptive Gradient Methods for Strongly-Convex Functions},
author = {Deepak Gouda and Hassan Naveed and Salil Kamath},
journal= {arXiv preprint arXiv:2306.06613},
year = {2023}
}