Stochastic gradient descent algorithms for strongly convex functions at O(1/T) convergence rates
Machine Learning
2013-05-13 v1 Artificial Intelligence
Abstract
With a weighting scheme proportional to t, a traditional stochastic gradient descent (SGD) algorithm achieves a high probability convergence rate of O({\kappa}/T) for strongly convex functions, instead of O({\kappa} ln(T)/T). We also prove that an accelerated SGD algorithm also achieves a rate of O({\kappa}/T).
Keywords
Cite
@article{arxiv.1305.2218,
title = {Stochastic gradient descent algorithms for strongly convex functions at O(1/T) convergence rates},
author = {Shenghuo Zhu},
journal= {arXiv preprint arXiv:1305.2218},
year = {2013}
}