Greedy Quasi-Newton Methods with Explicit Superlinear Convergence
Abstract
In this paper, we study greedy variants of quasi-Newton methods. They are based on the updating formulas from a certain subclass of the Broyden family. In particular, this subclass includes the well-known DFP, BFGS and SR1 updates. However, in contrast to the classical quasi-Newton methods, which use the difference of successive iterates for updating the Hessian approximations, our methods apply basis vectors, greedily selected so as to maximize a certain measure of progress. For greedy quasi-Newton methods, we establish an explicit non-asymptotic bound on their rate of local superlinear convergence, which contains a contraction factor, depending on the square of the iteration counter. We also show that these methods produce Hessian approximations whose deviation from the exact Hessians linearly convergences to zero.
Cite
@article{arxiv.2002.00657,
title = {Greedy Quasi-Newton Methods with Explicit Superlinear Convergence},
author = {Anton Rodomanov and Yurii Nesterov},
journal= {arXiv preprint arXiv:2002.00657},
year = {2021}
}