Stochastic Recursive Gradient Algorithm for Nonconvex Optimization
Machine Learning
2017-05-23 v1 Machine Learning
Optimization and Control
Abstract
In this paper, we study and analyze the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses. We provide a sublinear convergence rate (to stationary points) for general nonconvex functions and a linear convergence rate for gradient dominated functions, both of which have some advantages compared to other modern stochastic gradient algorithms for nonconvex losses.
Cite
@article{arxiv.1705.07261,
title = {Stochastic Recursive Gradient Algorithm for Nonconvex Optimization},
author = {Lam M. Nguyen and Jie Liu and Katya Scheinberg and Martin Takáč},
journal= {arXiv preprint arXiv:1705.07261},
year = {2017}
}