Double Descent: Understanding Linear Model Estimation of Nonidentifiable Parameters and a Model for Overfitting
Machine Learning
2025-09-19 v2 Machine Learning
Statistics Theory
Statistics Theory
Abstract
We consider ordinary least squares estimation and variations on least squares estimation such as penalized (regularized) least squares and spectral shrinkage estimates for problems with p > n and associated problems with prediction of new observations. After the introduction of Section 1, Section 2 examines a number of commonly used estimators for p > n. Section 3 introduces prediction with p > n. Section 4 introduces notational changes to facilitate discussion of overfitting and Section 5 illustrates the phenomenon of double descent. We conclude with some final comments.
Cite
@article{arxiv.2408.13235,
title = {Double Descent: Understanding Linear Model Estimation of Nonidentifiable Parameters and a Model for Overfitting},
author = {Ronald Christensen},
journal= {arXiv preprint arXiv:2408.13235},
year = {2025}
}