English
Related papers

Related papers: Benign overfitting without concentration

200 papers

Evaluating treatments received by one population for application to a different target population of scientific interest is a central problem in causal inference from observational studies. We study the minimax linear estimator of the…

Statistics Theory · Mathematics 2021-03-01 David A. Hirshberg , Arian Maleki , Jose R. Zubizarreta

Empirical research typically involves a robustness-efficiency tradeoff. A researcher seeking to estimate a scalar parameter can invoke strong assumptions to motivate a restricted estimator that is precise but may be heavily biased, or they…

Econometrics · Economics 2025-09-17 Timothy B. Armstrong , Patrick Kline , Liyang Sun

Complementary-label learning is a weakly supervised learning problem in which each training example is associated with one or multiple complementary labels indicating the classes to which it does not belong. Existing consistent approaches…

Machine Learning · Computer Science 2024-10-14 Wei Wang , Takashi Ishida , Yu-Jie Zhang , Gang Niu , Masashi Sugiyama

We study random design linear regression with no assumptions on the distribution of the covariates and with a heavy-tailed response variable. In this distribution-free regression setting, we show that boundedness of the conditional second…

Statistics Theory · Mathematics 2022-02-25 Jaouad Mourtada , Tomas Vaškevičius , Nikita Zhivotovskiy

Overfitting is a phenomenon that occurs when a machine learning model is trained for too long and focused too much on the exact fitness of the training samples to the provided training labels and cannot keep track of the predictive rules…

Machine Learning · Computer Science 2025-09-22 Nuri Korhan , Samet Bayram

Aligning powerful AI models on tasks that surpass human evaluation capabilities is the central problem of \textbf{superalignment}. To address this problem, weak-to-strong generalization aims to elicit the capabilities of strong models…

Machine Learning · Computer Science 2025-03-07 Junhao Shi , Qinyuan Cheng , Zhaoye Fei , Yining Zheng , Qipeng Guo , Xipeng Qiu

This study investigates the critical problem of overfitting in deep learning models applied to histopathology image analysis. We show that simply adopting and fine-tuning large-scale models designed for natural image analysis often leads to…

Image and Video Processing · Electrical Eng. & Systems 2025-06-23 Saghir Alfasly , Ghazal Alabtah , H. R. Tizhoosh

We consider linear models where $d$ potential causes $X_1,...,X_d$ are correlated with one target quantity $Y$ and propose a method to infer whether the association is causal or whether it is an artifact caused by overfitting or hidden…

Machine Learning · Statistics 2018-03-05 Dominik Janzing , Bernhard Schoelkopf

A simple test is proposed for examining the correctness of a given completely specified response function against unspecified general alternatives in the context of univariate regression. The usual diagnostic tools based on residuals plots…

Methodology · Statistics 2010-04-27 Jean-Baptiste Aubin , Samuela Leoni-Aubin

Semi-supervised learning by self-training heavily relies on pseudo-label selection (PLS). The selection often depends on the initial model fit on labeled data. Early overfitting might thus be propagated to the final model by selecting…

Machine Learning · Statistics 2023-06-27 Julian Rodemann , Jann Goschenhofer , Emilio Dorigatti , Thomas Nagler , Thomas Augustin

For many important problems the quantity of interest is an unknown function of the parameters, which is a random vector with known statistics. Since the dependence of the output on this random vector is unknown, the challenge is to identify…

Machine Learning · Statistics 2021-04-28 Themistoklis P. Sapsis

We study hypothesis testing for penalized estimators in settings where the full marginal distribution of a multivariate response is difficult to specify, such as longitudinal data with correlated measurements or high-dimensional…

Methodology · Statistics 2026-04-08 Jing Zhou , Zhe Zhang

We study the convergence of the predictive surface of regression trees and forests. To support our analysis we introduce a notion of adaptive concentration for regression trees. This approach breaks tree training into a model selection…

Statistics Theory · Mathematics 2016-05-03 Stefan Wager , Guenther Walther

Due to its strong interpretability, linear regression is widely used in social science, from which significance test provides the significance level of models or coefficients in the traditional statistical inference. However, linear…

Machine Learning · Computer Science 2020-06-08 Jiaye Teng , Yang Yuan

Neural network models and deep models are one of the leading and state of the art models in machine learning. Most successful deep neural models are the ones with many layers which highly increases their number of parameters. Training such…

Machine Learning · Computer Science 2018-07-17 Soufiane Belharbi

Modern machine learning classifiers often exhibit vanishing classification error on the training set. They achieve this by learning nonlinear representations of the inputs that maps the data into linearly separable classes. Motivated by…

Statistics Theory · Mathematics 2023-03-23 Andrea Montanari , Feng Ruan , Youngtak Sohn , Jun Yan

Transformers have the capacity to act as supervised learning algorithms: by properly encoding a set of labeled training ("in-context") examples and an unlabeled test example into an input sequence of vectors of the same dimension, the…

Machine Learning · Computer Science 2024-12-16 Spencer Frei , Gal Vardi

We emphasize that it is possible to improve the principle of unbiased risk estimation for model selection by addressing excess risk deviations in the design of penalization procedures. Indeed, we propose a modification of Akaike's…

Statistics Theory · Mathematics 2018-07-23 Adrien Saumard , Fabien Navarro

Importance weighting is a standard tool for correcting distribution shift, but its statistical behavior under target shift -- where the label distribution changes between training and testing while the conditional distribution of inputs…

Machine Learning · Statistics 2026-03-04 Davit Gogolashvili

Addressing selection bias in latent variable causal discovery is important yet underexplored, largely due to a lack of suitable statistical tools: While various tools beyond basic conditional independencies have been developed to handle…

Machine Learning · Computer Science 2025-12-15 Haoyue Dai , Yiwen Qiu , Ignavier Ng , Xinshuai Dong , Peter Spirtes , Kun Zhang