Second-order Guarantees of Distributed Gradient Algorithms
Abstract
We consider distributed smooth nonconvex unconstrained optimization over networks, modeled as a connected graph. We examine the behavior of distributed gradient-based algorithms near strict saddle points. Specifically, we establish that (i) the renowned Distributed Gradient Descent (DGD) algorithm likely converges to a neighborhood of a Second-order Stationary (SoS) solution; and (ii) the more recent class of distributed algorithms based on gradient tracking--implementable also over digraphs--likely converges to exact SoS solutions, thus avoiding (strict) saddle-points. Furthermore, new convergence rate results to first-order critical points is established for the latter class of algorithms.
Cite
@article{arxiv.1809.08694,
title = {Second-order Guarantees of Distributed Gradient Algorithms},
author = {Amir Daneshmand and Gesualdo Scutari and Vyacheslav Kungurtsev},
journal= {arXiv preprint arXiv:1809.08694},
year = {2020}
}
Comments
Final version, to appear on SIAM J. on Optimization