GPU-Parallelizable Randomized Sketch-and-Precondition for Linear Regression using Sparse Sign Sketches
Abstract
A litany of theoretical and numerical results have established the sketch-and-precondition paradigm as a powerful approach to solving large linear regression problems in standard computing environments. Perhaps surprisingly, much less work has been done on understanding how sketch-and-precondition performs on graphics processing unit (GPU) systems. We address this gap by benchmarking an implementation of sketch-and-precondition based on sparse sign-sketches on single and multi-GPU systems. In doing so, we describe a novel, easily parallelized, rejection-sampling based method for generating sparse sign sketches. Our approach, which is particularly well-suited for GPUs, is easily adapted to a variety of computing environments. Taken as a whole, our numerical experiments indicate that sketch-and-precondition with sparse sign sketches is particularly well-suited for GPUs, and may be suitable for use in black-box least-squares solvers.
Cite
@article{arxiv.2506.03070,
title = {GPU-Parallelizable Randomized Sketch-and-Precondition for Linear Regression using Sparse Sign Sketches},
author = {Tyler Chen and Pradeep Niroula and Archan Ray and Pragna Subrahmanya and Marco Pistoia and Niraj Kumar},
journal= {arXiv preprint arXiv:2506.03070},
year = {2025}
}