Accurate, Fast and Scalable Kernel Ridge Regression on Parallel and Distributed Systems
Abstract
We propose two new methods to address the weak scaling problems of KRR: the Balanced KRR (BKRR) and K-means KRR (KKRR). These methods consider alternative ways to partition the input dataset into p different parts, generating p different models, and then selecting the best model among them. Compared to a conventional implementation, KKRR2 (optimized version of KKRR) improves the weak scaling efficiency from 0.32% to 38% and achieves a 591times speedup for getting the same accuracy by using the same data and the same hardware (1536 processors). BKRR2 (optimized version of BKRR) achieves a higher accuracy than the current fastest method using less training time for a variety of datasets. For the applications requiring only approximate solutions, BKRR2 improves the weak scaling efficiency to 92% and achieves 3505 times speedup (theoretical speedup: 4096 times).
Cite
@article{arxiv.1805.00569,
title = {Accurate, Fast and Scalable Kernel Ridge Regression on Parallel and Distributed Systems},
author = {Yang You and James Demmel and Cho-Jui Hsieh and Richard Vuduc},
journal= {arXiv preprint arXiv:1805.00569},
year = {2018}
}
Comments
This paper has been accepted by ACM International Conference on Supercomputing (ICS) 2018