English
Related papers

Related papers: Benign overfitting in ridge regression

200 papers

Modern machine learning often operates in the regime where the number of parameters is much higher than the number of data points, with zero training loss and yet good generalization, thereby contradicting the classical bias-variance…

Machine Learning · Statistics 2021-02-08 Zhu Li , Weijie Su , Dino Sejdinovic

Understanding how overparameterized neural networks generalize despite perfect interpolation of noisy training data is a fundamental question. Mallinar et. al. 2022 noted that neural networks seem to often exhibit ``tempered overfitting'',…

Machine Learning · Computer Science 2024-03-25 Nirmit Joshi , Gal Vardi , Nathan Srebro

Recent research in neural networks and machine learning suggests that using many more parameters than strictly required by the initial complexity of a regression problem can result in more accurate or faster-converging models -- contrary to…

Machine Learning · Computer Science 2023-05-18 Arthur Castello B. de Oliveira , Milad Siami , Eduardo D. Sontag

Transfer learning is a critical part of real-world machine learning deployments and has been extensively studied in experimental works with overparameterized neural networks. However, even in the simplest setting of linear regression a…

Machine Learning · Computer Science 2024-08-28 Neil Mallinar , Austin Zane , Spencer Frei , Bin Yu

We study theoretical predictive performance of ridge and ridge-less least-squares regression when covariate vectors arise from evaluating $p$ random, means-square continuous functions over a latent metric space at $n$ random and unobserved…

Machine Learning · Statistics 2025-08-20 Andrew Jones , Nick Whiteley

In our era of enormous neural networks, empirical progress has been driven by the philosophy that more is better. Recent deep learning practice has found repeatedly that larger model size, more data, and more computation (resulting in lower…

Machine Learning · Computer Science 2024-05-17 James B. Simon , Dhruva Karkada , Nikhil Ghosh , Mikhail Belkin

We examine the necessity of interpolation in overparameterized models, that is, when achieving optimal predictive risk in machine learning problems requires (nearly) interpolating the training data. In particular, we consider simple…

Machine Learning · Statistics 2022-06-17 Chen Cheng , John Duchi , Rohith Kuditipudi

Recently, deep neural networks have been found to nearly interpolate training data but still generalize well in various applications. To help understand such a phenomenon, it has been of interest to analyze the ridge estimator and its…

Statistics Theory · Mathematics 2024-05-03 Libin Liang , Zhiqiang Tan

Classical wisdom suggests that estimators should avoid fitting noise to achieve good generalization. In contrast, modern overparameterized models can yield small test error despite interpolating noise -- a phenomenon often called "benign…

Machine Learning · Statistics 2023-03-02 Michael Aerni , Marco Milanta , Konstantin Donhauser , Fanny Yang

Overparametrized neural networks tend to perfectly fit noisy training data yet generalize well on test data. Inspired by this empirical observation, recent work has sought to understand this phenomenon of benign overfitting or harmless…

Machine Learning · Statistics 2022-02-23 Andrew D. McRae , Santhosh Karnik , Mark A. Davenport , Vidya Muthukumar

In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still…

Statistics Theory · Mathematics 2020-07-27 Tengyuan Liang , Alexander Rakhlin

From benign overfitting in overparameterized models to rich power-law scalings in performance, simple ridge regression displays surprising behaviors sometimes thought to be limited to deep neural networks. This balance of phenomenological…

Machine Learning · Statistics 2026-05-08 Alexander Atanasov , Jacob A. Zavatone-Veth , Cengiz Pehlevan

The remarkable practical success of deep learning has revealed some major surprises from a theoretical perspective. In particular, simple gradient methods easily find near-optimal solutions to non-convex optimization problems, and despite…

Statistics Theory · Mathematics 2021-03-17 Peter L. Bartlett , Andrea Montanari , Alexander Rakhlin

In deep learning, often the training process finds an interpolator (a solution with 0 training loss), but the test loss is still low. This phenomenon, known as benign overfitting, is a major mystery that received a lot of recent attention.…

Machine Learning · Computer Science 2023-05-29 Mo Zhou , Rong Ge

Establishing a theoretical analysis that explains why deep learning can outperform shallow learning such as kernel methods is one of the biggest issues in the deep learning literature. Towards answering this question, we evaluate excess…

Machine Learning · Statistics 2020-12-08 Taiji Suzuki , Shunta Akiyama

Modern machine learning models with a large number of parameters often generalize well despite perfectly interpolating noisy training data - a phenomenon known as benign overfitting. A foundational explanation for this in linear…

Machine Learning · Statistics 2025-11-18 Yuta Kondo

Modern deep learning models are usually highly over-parameterized so that they can overfit the training data. Surprisingly, such overfitting neural networks can usually still achieve high prediction accuracy. To study this "benign…

Machine Learning · Computer Science 2023-10-04 Xuran Meng , Difan Zou , Yuan Cao

In the context of neural network models, overparametrization refers to the phenomena whereby these models appear to generalize well on the unseen data, even though the number of parameters significantly exceeds the sample sizes, and the…

Machine Learning · Statistics 2020-03-25 Matt Emschwiller , David Gamarnik , Eren C. Kızıldağ , Ilias Zadik

We study the overfitting behavior of fully connected deep Neural Networks (NNs) with binary weights fitted to perfectly classify a noisy training set. We consider interpolation using both the smallest NN (having the minimal number of…

Machine Learning · Computer Science 2024-10-28 Itamar Harel , William M. Hoza , Gal Vardi , Itay Evron , Nathan Srebro , Daniel Soudry

The literature on "benign overfitting" in overparameterized models has been mostly restricted to regression or binary classification; however, modern machine learning operates in the multiclass setting. Motivated by this discrepancy, we…

Machine Learning · Statistics 2023-07-13 Ke Wang , Vidya Muthukumar , Christos Thrampoulidis