We study the use of linear regression for multiclass classification in the over-parametrized regime where some of the training data is mislabeled. In such scenarios it is necessary to add an explicit regularization term, λf(w), for some convex function f(⋅), to avoid overfitting the mislabeled data. In our analysis, we assume that the data is sampled from a Gaussian Mixture Model with equal class sizes, and that a proportion c of the training labels is corrupted for each class. Under these assumptions, we prove that the best classification performance is achieved when f(⋅)=∥⋅∥22 and λ→∞. We then proceed to analyze the classification errors for f(⋅)=∥⋅∥1 and f(⋅)=∥⋅∥∞ in the large λ regime and notice that it is often possible to find sparse and one-bit solutions, respectively, that perform almost as well as the one corresponding to f(⋅)=∥⋅∥22.
@article{arxiv.2402.10474,
title = {One-Bit Quantization and Sparsification for Multiclass Linear Classification with Strong Regularization},
author = {Reza Ghane and Danil Akhtiamov and Babak Hassibi},
journal= {arXiv preprint arXiv:2402.10474},
year = {2024}
}