Online Learning, Stability, and Stochastic Gradient Descent
Machine Learning
2015-03-19 v3
Abstract
In batch learning, stability together with existence and uniqueness of the solution corresponds to well-posedness of Empirical Risk Minimization (ERM) methods; recently, it was proved that CV_loo stability is necessary and sufficient for generalization and consistency of ERM. In this note, we introduce CV_on stability, which plays a similar note in online learning. We show that stochastic gradient descent (SDG) with the usual hypotheses is CVon stable and we then discuss the implications of CV_on stability for convergence of SGD.
Keywords
Cite
@article{arxiv.1105.4701,
title = {Online Learning, Stability, and Stochastic Gradient Descent},
author = {Tomaso Poggio and Stephen Voinea and Lorenzo Rosasco},
journal= {arXiv preprint arXiv:1105.4701},
year = {2015}
}
Comments
11 pages