English

Stochastic subgradient method converges at the rate $O(k^{-1/4})$ on weakly convex functions

Optimization and Control 2018-02-21 v3 Machine Learning

Abstract

We prove that the proximal stochastic subgradient method, applied to a weakly convex problem, drives the gradient of the Moreau envelope to zero at the rate O(k1/4)O(k^{-1/4}). As a consequence, we resolve an open question on the convergence rate of the proximal stochastic gradient method for minimizing the sum of a smooth nonconvex function and a convex proximable function.

Keywords

Cite

@article{arxiv.1802.02988,
  title  = {Stochastic subgradient method converges at the rate $O(k^{-1/4})$ on weakly convex functions},
  author = {Damek Davis and Dmitriy Drusvyatskiy},
  journal= {arXiv preprint arXiv:1802.02988},
  year   = {2018}
}

Comments

12 pages