Conjugate Gradients and Accelerated Methods Unified: The Approximate Duality Gap View
Abstract
This note provides a novel, simple analysis of the method of conjugate gradients for the minimization of convex quadratic functions. In contrast with standard arguments, our proof is entirely self-contained and does not rely on the existence of Chebyshev polynomials. Another advantage of our development is that it clarifies the relation between the method of conjugate gradients and general accelerated methods for smooth minimization by unifying their analyses within the framework of the Approximate Duality Gap Technique that was introduced by the authors.
Cite
@article{arxiv.1907.00289,
title = {Conjugate Gradients and Accelerated Methods Unified: The Approximate Duality Gap View},
author = {Jelena Diakonikolas and Lorenzo Orecchia},
journal= {arXiv preprint arXiv:1907.00289},
year = {2020}
}
Comments
8 pages. v1 -> v2: corrected a reference to the paper with Nemirovski acceleration with line search. v2 -> v3: updated affiliations, corrected a few typos on p.7 and added an acknowledgement