A frequency-domain analysis of inexact gradient methods
Abstract
We study robustness properties of some iterative gradient-based methods for strongly convex functions, as well as for the larger class of functions with sector-bounded gradients, under a relative error model. Proofs of the corresponding convergence rates are based on frequency-domain criteria for the stability of nonlinear systems. Applications are given to inexact versions of gradient descent and the Triple Momentum Method. To further emphasize the usefulness of frequency-domain methods, we derive improved analytic bounds for the convergence rate of Nesterov's accelerated method (in the exact setting) on strongly convex functions.
Cite
@article{arxiv.1912.13494,
title = {A frequency-domain analysis of inexact gradient methods},
author = {Oran Gannot},
journal= {arXiv preprint arXiv:1912.13494},
year = {2021}
}
Comments
42 pages; corrections and additional applications to accelerated methods. To appear in Mathematical Programming