A Parallel SGD method with Strong Convergence
Machine Learning
2013-11-05 v1 Distributed, Parallel, and Cluster Computing
Abstract
This paper proposes a novel parallel stochastic gradient descent (SGD) method that is obtained by applying parallel sets of SGD iterations (each set operating on one node using the data residing in it) for finding the direction in each iteration of a batch descent method. The method has strong convergence properties. Experiments on datasets with high dimensional feature spaces show the value of this method.
Cite
@article{arxiv.1311.0636,
title = {A Parallel SGD method with Strong Convergence},
author = {Dhruv Mahajan and S. Sathiya Keerthi and S. Sundararajan and Leon Bottou},
journal= {arXiv preprint arXiv:1311.0636},
year = {2013}
}