English
Related papers

Related papers: Benign overfitting without concentration

200 papers

Overparametrized neural networks tend to perfectly fit noisy training data yet generalize well on test data. Inspired by this empirical observation, recent work has sought to understand this phenomenon of benign overfitting or harmless…

Machine Learning · Statistics 2022-02-23 Andrew D. McRae , Santhosh Karnik , Mark A. Davenport , Vidya Muthukumar

It is well known that popular optimization techniques can lead to overfitting or even a lack of convergence altogether; thus, practitioners often utilize ad hoc regularization terms added to the energy functional. When carefully crafted,…

Computer Vision and Pattern Recognition · Computer Science 2020-08-18 Michael Bao , David Hyde , Xinru Hua , Ronald Fedkiw

Classical wisdom suggests that estimators should avoid fitting noise to achieve good generalization. In contrast, modern overparameterized models can yield small test error despite interpolating noise -- a phenomenon often called "benign…

Machine Learning · Statistics 2023-03-02 Michael Aerni , Marco Milanta , Konstantin Donhauser , Fanny Yang

In this work, we investigate the behavior of ridge regression in an overparameterized binary classification task. We assume examples are drawn from (anisotropic) class-conditional cluster distributions with opposing means and we allow for…

Machine Learning · Statistics 2025-03-12 Alexander Tsigler , Luiz F. O. Chamon , Spencer Frei , Peter L. Bartlett

The repeated community-wide reuse of test sets in popular benchmark problems raises doubts about the credibility of reported test-error rates. Verifying whether a learned model is overfitted to a test set is challenging as independent test…

Machine Learning · Computer Science 2019-11-15 Roman Werpachowski , András György , Csaba Szepesvári

We consider models of Bayesian inference of signals with vectorial components of finite dimensionality. We show that, under a proper perturbation, these models are replica symmetric in the sense that the overlap matrix concentrates. The…

Information Theory · Computer Science 2020-01-27 Jean Barbier

The local Rademacher complexity framework is one of the most successful general-purpose toolboxes for establishing sharp excess risk bounds for statistical estimators based on the framework of empirical risk minimization. Applying this…

Statistics Theory · Mathematics 2022-02-24 Varun Kanade , Patrick Rebeschini , Tomas Vaskevicius

Overfitting in deep learning has been the focus of a number of recent works, yet its exact impact on the behavior of neural networks is not well understood. This study analyzes overfitting by examining how the distribution of logits alters…

Machine Learning · Computer Science 2019-10-02 Zeju Li , Konstantinos Kamnitsas , Ben Glocker

Recent research in neural networks and machine learning suggests that using many more parameters than strictly required by the initial complexity of a regression problem can result in more accurate or faster-converging models -- contrary to…

Machine Learning · Computer Science 2023-05-18 Arthur Castello B. de Oliveira , Milad Siami , Eduardo D. Sontag

The widely observed 'benign overfitting phenomenon' in the neural network literature raises the challenge to the 'bias-variance trade-off' doctrine in the statistical learning theory. Since the generalization ability of the 'lazy trained'…

Machine Learning · Computer Science 2023-09-26 Yicheng Li , Haobo Zhang , Qian Lin

Object counting models suffer when deployed across domains with differing density variety, since density shifts are inherently task-relevant and violate standard domain adaptation assumptions. To address this, we propose a theoretical…

Computer Vision and Pattern Recognition · Computer Science 2025-11-03 Zhuonan Liang , Dongnan Liu , Jianan Fan , Yaxuan Song , Qiang Qu , Runnan Chen , Yu Yao , Peng Fu , Weidong Cai

We study the prevalent problem when a test distribution differs from the training distribution. We consider a setting where our training set consists of a small number of sample domains, but where we have many samples in each domain. Our…

Machine Learning · Computer Science 2011-05-05 Dean Foster , Sham Kakade , Ruslan Salakhutdinov

We consider the problem of predicting as well as the best linear combination of d given functions in least squares regression under L^\infty constraints on the linear combination. When the input distribution is known, there already exists…

Statistics Theory · Mathematics 2011-09-14 Jean-Yves Audibert , Olivier Catoni

Despite substantial progress in promoting fairness in high-stake applications using machine learning models, existing methods often modify the training process, such as through regularizers or other interventions, but lack formal guarantees…

Machine Learning · Computer Science 2025-06-10 Firas Laakom , Haobo Chen , Jürgen Schmidhuber , Yuheng Bu

For the last two decades, high-dimensional data and methods have proliferated throughout the literature. Yet, the classical technique of linear regression has not lost its usefulness in applications. In fact, many high-dimensional…

Statistics Theory · Mathematics 2021-05-18 Arun Kumar Kuchibhotla , Lawrence D. Brown , Andreas Buja , Edward I. George , Linda Zhao

Although conformal prediction provides robust marginal coverage guarantees, achieving reliable conditional coverage for specific inputs remains challenging. While exact distribution-free conditional coverage is impossible with finite…

Machine Learning · Computer Science 2026-05-20 Qianyi Chen , Bo Li

From a systems biology perspective the majority of cancer models, although interesting and providing a qualitative explanation of some problems, have a major disadvantage in that they usually miss a genuine connection with experimental…

Statistics Theory · Mathematics 2023-05-25 Zuzanna Szymańska , Jakub Skrzeczkowski , Błażej Miasojedow , Piotr Gwiazda

Class imbalance poses a challenge for developing unbiased, accurate predictive models. In particular, in image segmentation neural networks may overfit to the foreground samples from small structures, which are often heavily…

Computer Vision and Pattern Recognition · Computer Science 2021-02-23 Zeju Li , Konstantinos Kamnitsas , Ben Glocker

We study theoretical predictive performance of ridge and ridge-less least-squares regression when covariate vectors arise from evaluating $p$ random, means-square continuous functions over a latent metric space at $n$ random and unobserved…

Machine Learning · Statistics 2025-08-20 Andrew Jones , Nick Whiteley

When using machine learning for imbalanced binary classification problems, it is common to subsample the majority class to create a (more) balanced training dataset. This biases the model's predictions because the model learns from data…

Machine Learning · Computer Science 2025-11-03 Nathan Phelps , Daniel J. Lizotte , Douglas G. Woolford