On the global convergence of randomized coordinate gradient descent for non-convex optimization
Optimization and Control
2022-12-01 v2 Numerical Analysis
Dynamical Systems
Numerical Analysis
Abstract
In this work, we analyze the global convergence property of coordinate gradient descent with random choice of coordinates and stepsizes for non-convex optimization problems. Under generic assumptions, we prove that the algorithm iterate will almost surely escape strict saddle points of the objective function. As a result, the algorithm is guaranteed to converge to local minima if all saddle points are strict. Our proof is based on viewing coordinate descent algorithm as a nonlinear random dynamical system and a quantitative finite block analysis of its linearization around saddle points.
Cite
@article{arxiv.2101.01323,
title = {On the global convergence of randomized coordinate gradient descent for non-convex optimization},
author = {Ziang Chen and Yingzhou Li and Jianfeng Lu},
journal= {arXiv preprint arXiv:2101.01323},
year = {2022}
}