English
Related papers

Related papers: Benign Overfitting in Linear Regression

200 papers

Existing large-dimensional theory for spectral algorithms resolves either the optimally tuned point or the interpolation limit, but leaves the under-regularized regime unexplored. We study the learning curve and benign overfitting of…

Machine Learning · Statistics 2026-04-28 Weihao Lu , Qian Lin , Yingcun Xia , Dongming Huang

Avoiding overfitting is a central challenge in machine learning, yet many large neural networks readily achieve zero training loss. This puzzling contradiction necessitates new approaches to the study of overfitting. Here we quantify…

Information Theory · Computer Science 2022-10-13 Vudtiwat Ngampruetikorn , David J. Schwab

We investigate two causes for adversarial vulnerability in deep neural networks: bad data and (poorly) trained models. When trained with SGD, deep neural networks essentially achieve zero training error, even in the presence of label noise,…

Machine Learning · Computer Science 2020-07-09 Amartya Sanyal , Puneet K Dokania , Varun Kanade , Philip H. S. Torr

The problem of benign overfitting asks whether it is possible for a model to perfectly fit noisy training data and still generalize well. We study benign overfitting in two-layer leaky ReLU networks trained with the hinge loss on a binary…

Machine Learning · Computer Science 2024-10-04 Kedar Karhadkar , Erin George , Michael Murray , Guido Montúfar , Deanna Needell

Meta-learning has arisen as a successful method for improving training performance by training over many similar tasks, especially with deep neural networks (DNNs). However, the theoretical understanding of when and why overparameterized…

Machine Learning · Computer Science 2023-04-11 Peizhong Ju , Yingbin Liang , Ness B. Shroff

Overfitting describes a machine learning phenomenon where the model fits too closely to the training data, resulting in poor generalization. While this occurrence is thoroughly documented for many forms of supervised learning, it is not…

Machine Learning · Computer Science 2024-08-23 Zachary Rabin , Jim Davis , Benjamin Lewis , Matthew Scherreik

We study the benign overfitting theory in the prediction of the conditional average treatment effect (CATE), with linear regression models. As the development of machine learning for causal inference, a wide range of large-scale models for…

Econometrics · Economics 2022-02-17 Masahiro Kato , Masaaki Imaizumi

In the context of neural network models, overparametrization refers to the phenomena whereby these models appear to generalize well on the unseen data, even though the number of parameters significantly exceeds the sample sizes, and the…

Machine Learning · Statistics 2020-03-25 Matt Emschwiller , David Gamarnik , Eren C. Kızıldağ , Ilias Zadik

Learned classifiers should often possess certain invariance properties meant to encourage fairness, robustness, or out-of-distribution generalization. However, multiple recent works empirically demonstrate that common invariance-inducing…

Machine Learning · Computer Science 2024-07-04 Yoav Wald , Gal Yona , Uri Shalit , Yair Carmon

We obtain a sufficient condition for benign overfitting of linear regression problem. Our result does not rely on concentration argument but on small-ball assumption and thus can holds in heavy-tailed case. The basic idea is to establish a…

Statistics Theory · Mathematics 2021-01-05 Zong Shang

Deep learning is renowned for its theory-practice gap, whereby principled theory typically fails to provide much beneficial guidance for implementation in practice. This has been highlighted recently by the benign overfitting phenomenon:…

Machine Learning · Statistics 2023-11-14 Liam Hodgkinson , Chris van der Heide , Robert Salomone , Fred Roosta , Michael W. Mahoney

In a regression context, when the relevant subset of explanatory variables is uncertain, it is common to use a data-driven model selection procedure. Classical linear model theory, applied naively to the selected sub-model, may not be valid…

Statistics Theory · Mathematics 2017-12-08 Liang Hong , Todd A. Kuffner , Ryan Martin

Why heavily parameterized neural networks (NNs) do not overfit the data is an important long standing open question. We propose a phenomenological model of the NN training to explain this non-overfitting puzzle. Our linear frequency…

Machine Learning · Computer Science 2021-05-26 Yaoyu Zhang , Tao Luo , Zheng Ma , Zhi-Qin John Xu

A main puzzle of deep networks revolves around the absence of overfitting despite large overparametrization and despite the large capacity demonstrated by zero training error on randomly labeled data. In this note, we show that the dynamics…

Overfitting in deep learning has been the focus of a number of recent works, yet its exact impact on the behavior of neural networks is not well understood. This study analyzes overfitting by examining how the distribution of logits alters…

Machine Learning · Computer Science 2019-10-02 Zeju Li , Konstantinos Kamnitsas , Ben Glocker

Neural network models and deep models are one of the leading and state of the art models in machine learning. Most successful deep neural models are the ones with many layers which highly increases their number of parameters. Training such…

Machine Learning · Computer Science 2018-07-17 Soufiane Belharbi

A regression model with more parameters than data points in the training data is overparametrized and has the capability to interpolate the training data. Based on the classical bias-variance tradeoff expressions, it is commonly assumed…

Machine Learning · Computer Science 2023-04-18 Tomas McKelvey

In the linear regression model, the minimum l2-norm interpolant estimator has received much attention since it was proved to be consistent even though it fits noisy data perfectly under some condition on the covariance matrix $\Sigma$ of…

Statistics Theory · Mathematics 2023-09-25 Guillaume Lecué , Zong Shang

Overfitting is a phenomenon that occurs when a machine learning model is trained for too long and focused too much on the exact fitness of the training samples to the provided training labels and cannot keep track of the predictive rules…

Machine Learning · Computer Science 2025-09-22 Nuri Korhan , Samet Bayram

Understanding how overparameterized neural networks generalize despite perfect interpolation of noisy training data is a fundamental question. Mallinar et. al. 2022 noted that neural networks seem to often exhibit ``tempered overfitting'',…

Machine Learning · Computer Science 2024-03-25 Nirmit Joshi , Gal Vardi , Nathan Srebro