A Risk Comparison of Ordinary Least Squares vs Ridge Regression
Machine Learning
2013-06-03 v2
Abstract
We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a Principal Component Analysis) and then performs an ordinary (un-regularized) least squares regression in this subspace. This note shows that the risk of this ordinary least squares method is within a constant factor (namely 4) of the risk of ridge regression.
Keywords
Cite
@article{arxiv.1105.0875,
title = {A Risk Comparison of Ordinary Least Squares vs Ridge Regression},
author = {Paramveer S. Dhillon and Dean P. Foster and Sham M. Kakade and Lyle H. Ungar},
journal= {arXiv preprint arXiv:1105.0875},
year = {2013}
}
Comments
Appearing in JMLR 14, June 2013