无Lipschitz连续性条件下确定性与随机次梯度方法的收敛速率
最优化与控制
2018-02-28 v3 机器学习
摘要
我们将次梯度方法的经典收敛速率理论推广至非Lipschitz函数。对于确定性投影次梯度方法，我们给出了对任意在其极小点附近局部Lipschitz的凸函数的全局收敛速率。该方法基于Shor的经典次梯度分析，并意味着对具有Lipschitz或Hölder连续梯度函数的梯度下降标准收敛速率的推广。进一步，我们给出了随机投影次梯度方法在具有至多二次增长凸函数上的收敛速率，并在强凸性或较弱的二次下界条件下提升至。
引用
@article{arxiv.1712.04104,
title = {Convergence Rates for Deterministic and Stochastic Subgradient Methods Without Lipschitz Continuity},
author = {Benjamin Grimmer},
journal= {arXiv preprint arXiv:1712.04104},
year = {2018}
}
备注
Update 2/26/18: Major revision improving the convergence results to no longer need an exponential upper bound on function growth in the convex case. Now local Lipschitz continuity around a minimizer suffices for a global convergence rate. Update 12/21/17: Added three more references on weakening strong convexity and minorly changed some wording. 16 pages