Inertial Newton Algorithms Avoiding Strict Saddle Points
Optimization and Control
2024-02-13 v2 Machine Learning
Abstract
We study the asymptotic behavior of second-order algorithms mixing Newton's method and inertial gradient descent in non-convex landscapes. We show that, despite the Newtonian behavior of these methods, they almost always escape strict saddle points. We also evidence the role played by the hyper-parameters of these methods in their qualitative behavior near critical points. The theoretical results are supported by numerical illustrations.
Cite
@article{arxiv.2111.04596,
title = {Inertial Newton Algorithms Avoiding Strict Saddle Points},
author = {Camille Castera},
journal= {arXiv preprint arXiv:2111.04596},
year = {2024}
}