English
Related papers

Related papers: Foolish Crowds Support Benign Overfitting

200 papers

We obtain sharp oracle inequalities for the empirical risk minimization procedure in the regression model under the assumption that the target Y and the model F are subgaussian. The bound we obtain is sharp in the minimax sense if F is…

Statistics Theory · Mathematics 2016-09-20 Guillaume Lecué , Shahar Mendelson

Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss; yet surprisingly, they possess near-optimal prediction performance, contradicting classical learning theory. We…

Machine Learning · Statistics 2021-06-08 Zhu Li , Zhi-Hua Zhou , Arthur Gretton

We study the overfitting behavior of fully connected deep Neural Networks (NNs) with binary weights fitted to perfectly classify a noisy training set. We consider interpolation using both the smallest NN (having the minimal number of…

Machine Learning · Computer Science 2024-10-28 Itamar Harel , William M. Hoza , Gal Vardi , Itay Evron , Nathan Srebro , Daniel Soudry

Training neural network classifiers on datasets with label noise poses a risk of overfitting them to the noisy labels. To address this issue, researchers have explored alternative loss functions that aim to be more robust. The…

Machine Learning · Computer Science 2024-08-23 William Toner , Amos Storkey

In text classification, the problem of overfitting arises due to the high dimensionality, making regularization essential. Although classic regularizers provide sparsity, they fail to return highly accurate models. On the contrary,…

Machine Learning · Computer Science 2018-10-10 Konstantinos Skianis , Nikolaos Tziortziotis , Michalis Vazirgiannis

Variational inference with a factorized Gaussian posterior estimate is a widely used approach for learning parameters and hidden variables. Empirically, a regularizing effect can be observed that is poorly understood. In this work, we show…

Machine Learning · Computer Science 2019-09-04 Julius Kunze , Louis Kirsch , Hippolyt Ritter , David Barber

The marginal likelihood, also known as the evidence, is regarded as a mathematical embodiment of Occam's razor, enabling model selection that avoids overfitting. The evidence lower bound (ELBO) objective from variational inference has also…

Machine Learning · Statistics 2026-04-30 Ethan Harvey , Michael C. Hughes

This paper investigates robust versions of the general empirical risk minimization algorithm, one of the core techniques underlying modern statistical methods. Success of the empirical risk minimization is based on the fact that for a…

Machine Learning · Statistics 2019-10-17 Stanislav Minsker , Timothée Mathieu

We consider a deep neural network estimator based on empirical risk minimization with l_1-regularization. We derive a general bound for its excess risk in regression and classification (including multiclass), and prove that it is adaptively…

Statistics Theory · Mathematics 2023-11-16 Felix Abramovich

Counterfactual explanations are usually obtained by identifying the smallest change made to an input to change a prediction made by a fixed model (hereafter called sparse methods). Recent work, however, has revitalized an old insight: there…

Machine Learning · Computer Science 2020-06-24 Martin Pawelczyk , Klaus Broelemann , Gjergji Kasneci

In this paper, we investigate the theoretical guarantees of penalized $\lun$ minimization (also called Basis Pursuit Denoising or Lasso) in terms of sparsity pattern recovery (support and sign consistency) from noisy measurements with…

Information Theory · Computer Science 2011-09-13 Charles Dossal , Marie-Line Chabanol , Gabriel Peyré , Jalal Fadili

Benign overfitting, the phenomenon where interpolating models generalize well in the presence of noisy data, was first observed in neural network models trained with gradient descent. To better understand this empirical observation, we…

Machine Learning · Computer Science 2025-07-04 Spencer Frei , Niladri S. Chatterji , Peter L. Bartlett

In this paper we derive information theoretic performance bounds to sensing and reconstruction of sparse phenomena from noisy projections. We consider two settings: output noise models where the noise enters after the projection and input…

Information Theory · Computer Science 2011-12-22 Shuchin Aeron , Venkatesh Saligrama , Manqi Zhao

In presence of sparse noise we propose kernel regression for predicting output vectors which are smooth over a given graph. Sparse noise models the training outputs being corrupted either with missing samples or large perturbations. The…

Machine Learning · Statistics 2018-11-07 Arun Venkitaraman , Pascal Frossard , Saikat Chatterjee

Learning algorithms that divide the data into batches are prevalent in many machine-learning applications, typically offering useful trade-offs between computational efficiency and performance. In this paper, we examine the benefits of…

Machine Learning · Computer Science 2024-09-24 Shahar Stein Ioushua , Inbar Hasidim , Ofer Shayevitz , Meir Feder

The Ridgeless minimum $\ell_2$-norm interpolator in overparametrized linear regression has attracted considerable attention in recent years in both machine learning and statistics communities. While it seems to defy conventional wisdom that…

Statistics Theory · Mathematics 2026-01-21 Qiyang Han , Xiaocong Xu

In high-dimensional statistical inference, sparsity regularizations have shown advantages in consistency and convergence rates for coefficient estimation. We consider a generalized version of Sparse-Group Lasso which captures both…

Machine Learning · Statistics 2020-08-12 Xinyu Zhang

Consider estimating a structured signal $\mathbf{x}_0$ from linear, underdetermined and noisy measurements $\mathbf{y}=\mathbf{A}\mathbf{x}_0+\mathbf{z}$, via solving a variant of the lasso algorithm: $\hat{\mathbf{x}}=\arg\min_\mathbf{x}\{…

Optimization and Control · Mathematics 2014-01-28 Christos Thrampoulidis , Samet Oymak , Babak Hassibi

We assume the direct sum <A> o <B> for the signal subspace. As a result of post- measurement, a number of operational contexts presuppose the a priori knowledge of the LB -dimensional "interfering" subspace <B> and the goal is to estimate…

Applications · Statistics 2017-04-17 Guillaume Bouleux , Rémy Boyer

We present two sets of theoretical results on the grouped lasso with overlap of Jacob, Obozinski and Vert (2009) in the linear regression setting. This method allows for joint selection of predictors in sparse regression, allowing for…

Machine Learning · Statistics 2011-11-11 Daniel Percival