Distributed Gradient Descent with Coded Partial Gradient Computations
Machine Learning
2018-11-29 v1 Distributed, Parallel, and Cluster Computing
Information Theory
Signal Processing
math.IT
Machine Learning
Abstract
Coded computation techniques provide robustness against straggling servers in distributed computing, with the following limitations: First, they increase decoding complexity. Second, they ignore computations carried out by straggling servers; and they are typically designed to recover the full gradient, and thus, cannot provide a balance between the accuracy of the gradient and per-iteration completion time. Here we introduce a hybrid approach, called coded partial gradient computation (CPGC), that benefits from the advantages of both coded and uncoded computation schemes, and reduces both the computation time and decoding complexity.
Cite
@article{arxiv.1811.09271,
title = {Distributed Gradient Descent with Coded Partial Gradient Computations},
author = {Emre Ozfatura and Sennur Ulukus and Deniz Gunduz},
journal= {arXiv preprint arXiv:1811.09271},
year = {2018}
}