Global linear convergence of Newton's method without strong-convexity or Lipschitz gradients
Machine Learning
2018-06-04 v1 Optimization and Control
Machine Learning
Abstract
We show that Newton's method converges globally at a linear rate for objective functions whose Hessians are stable. This class of problems includes many functions which are not strongly convex, such as logistic regression. Our linear convergence result is (i) affine-invariant, and holds even if an (ii) approximate Hessian is used, and if the subproblems are (iii) only solved approximately. Thus we theoretically demonstrate the superiority of Newton's method over first-order methods, which would only achieve a sublinear rate under similar conditions.
Keywords
Cite
@article{arxiv.1806.00413,
title = {Global linear convergence of Newton's method without strong-convexity or Lipschitz gradients},
author = {Sai Praneeth Karimireddy and Sebastian U. Stich and Martin Jaggi},
journal= {arXiv preprint arXiv:1806.00413},
year = {2018}
}
Comments
19 pages