Factor-$\sqrt{2}$ Acceleration of Accelerated Gradient Methods
Abstract
The optimized gradient method (OGM) provides a factor- speedup upon Nesterov's celebrated accelerated gradient method in the convex (but non-strongly convex) setup. However, this improved acceleration mechanism has not been well understood; prior analyses of OGM relied on a computer-assisted proof methodology, so the proofs were opaque for humans despite being verifiable and correct. In this work, we present a new analysis of OGM based on a Lyapunov function and linear coupling. These analyses are developed and presented without the assistance of computers and are understandable by humans. Furthermore, we generalize OGM's acceleration mechanism and obtain a factor- speedup in other setups: acceleration with a simpler rational stepsize, the strongly convex setup, and the mirror descent setup.
Cite
@article{arxiv.2102.07366,
title = {Factor-$\sqrt{2}$ Acceleration of Accelerated Gradient Methods},
author = {Chanwoo Park and Jisun Park and Ernest K. Ryu},
journal= {arXiv preprint arXiv:2102.07366},
year = {2021}
}