Optimality of the final model found via Stochastic Gradient Descent
Machine Learning
2018-10-23 v1 Optimization and Control
Machine Learning
Abstract
We study convergence properties of Stochastic Gradient Descent (SGD) for convex objectives without assumptions on smoothness or strict convexity. We consider the question of establishing that with high probability the objective evaluated at the candidate minimizer returned by SGD is close to the minimal value of the objective. We compare this result concerning the final candidate minimzer (i.e. the final model parameters learned after all gradient steps) to the online learning techniques of [Zin03] that take a rolling average of the model parameters at the different steps of SGD.
Cite
@article{arxiv.1810.09418,
title = {Optimality of the final model found via Stochastic Gradient Descent},
author = {Andrea Schioppa},
journal= {arXiv preprint arXiv:1810.09418},
year = {2018}
}