Optimistic Rates for Learning with a Smooth Loss
Machine Learning
2012-11-27 v2
Abstract
We establish an excess risk bound of O(H R_n^2 + R_n \sqrt{H L*}) for empirical risk minimization with an H-smooth loss function and a hypothesis class with Rademacher complexity R_n, where L* is the best risk achievable by the hypothesis class. For typical hypothesis classes where R_n = \sqrt{R/n}, this translates to a learning rate of O(RH/n) in the separable (L*=0) case and O(RH/n + \sqrt{L^* RH/n}) more generally. We also provide similar guarantees for online and stochastic convex optimization with a smooth non-negative objective.
Cite
@article{arxiv.1009.3896,
title = {Optimistic Rates for Learning with a Smooth Loss},
author = {Nathan Srebro and Karthik Sridharan and Ambuj Tewari},
journal= {arXiv preprint arXiv:1009.3896},
year = {2012}
}