Distributed Gradient Methods for Nonconvex Optimization: Local and Global Convergence Guarantees
Optimization and Control
2020-09-17 v2
Abstract
The article discusses distributed gradient-descent algorithms for computing local and global minima in nonconvex optimization. For local optimization, we focus on distributed stochastic gradient descent (D-SGD)--a simple network-based variant of classical SGD. We discuss local minima convergence guarantees and explore the simple but critical role of the stable-manifold theorem in analyzing saddle-point avoidance. For global optimization, we discuss annealing-based methods in which slowly decaying noise is added to D-SGD. Conditions are discussed under which convergence to global minima is guaranteed. Numerical examples illustrate the key concepts in the paper.
Cite
@article{arxiv.2003.10309,
title = {Distributed Gradient Methods for Nonconvex Optimization: Local and Global Convergence Guarantees},
author = {Brian Swenson and Soummya Kar and H. Vincent Poor and José M. F. Moura and Aaron Jaech},
journal= {arXiv preprint arXiv:2003.10309},
year = {2020}
}