First Analysis of Local GD on Heterogeneous Data
Machine Learning
2020-03-19 v2 Distributed, Parallel, and Cluster Computing
Numerical Analysis
Numerical Analysis
Optimization and Control
Machine Learning
Abstract
We provide the first convergence analysis of local gradient descent for minimizing the average of smooth and convex but otherwise arbitrary functions. Problems of this form and local gradient descent as a solution method are of importance in federated learning, where each function is based on private data stored by a user on a mobile device, and the data of different users can be arbitrarily heterogeneous. We show that in a low accuracy regime, the method has the same communication complexity as gradient descent.
Cite
@article{arxiv.1909.04715,
title = {First Analysis of Local GD on Heterogeneous Data},
author = {Ahmed Khaled and Konstantin Mishchenko and Peter Richtárik},
journal= {arXiv preprint arXiv:1909.04715},
year = {2020}
}
Comments
NeurIPS 2019 Workshop on Federated Learning for Data Privacy and Confidentiality. 11 pages, 4 lemmas, 1 theorem