Accelerated Stochastic Gradient Descent for Minimizing Finite Sums
Machine Learning
2015-06-11 v2 Machine Learning
Abstract
We propose an optimization method for minimizing the finite sums of smooth convex functions. Our method incorporates an accelerated gradient descent (AGD) and a stochastic variance reduction gradient (SVRG) in a mini-batch setting. Unlike SVRG, our method can be directly applied to non-strongly and strongly convex problems. We show that our method achieves a lower overall complexity than the recently proposed methods that supports non-strongly convex problems. Moreover, this method has a fast rate of convergence for strongly convex problems. Our experiments show the effectiveness of our method.
Cite
@article{arxiv.1506.03016,
title = {Accelerated Stochastic Gradient Descent for Minimizing Finite Sums},
author = {Atsushi Nitanda},
journal= {arXiv preprint arXiv:1506.03016},
year = {2015}
}
Comments
[v2] corrected citation to proxSVRG, corrected typos in Figure 1(option2) and 3(R4 -> R3)