We develop a novel preconditioning method for ridge regression, based on recent linear sketching methods. By equipping Stochastic Variance Reduced Gradient (SVRG) with this preconditioning process, we obtain a significant speed-up relative to fast stochastic methods such as SVRG, SDCA and SAG.
Cite
@article{arxiv.1602.02350,
title = {Solving Ridge Regression using Sketched Preconditioned SVRG},
author = {Alon Gonen and Francesco Orabona and Shai Shalev-Shwartz},
journal= {arXiv preprint arXiv:1602.02350},
year = {2016}
}