Related papers: Ridge regularization for Mean Squared Error Reduct…
Variance estimation in the linear model when $p > n$ is a difficult problem. Standard least squares estimation techniques do not apply. Several variance estimators have been proposed in the literature, all with accompanying asymptotic…
Linearly parametrized models are widely used in control and signal processing, with the least-squares (LS) estimate being the archetypical solution. When the input is insufficiently exciting, the LS problem may be unsolvable or numerically…
Instrumental variable (IV) analysis is widely used in fields such as economics and epidemiology to address unobserved confounding and measurement error when estimating the causal effects of intermediate covariates on outcomes. However,…
A new risk bound is presented for the problem of convex/concave function estimation, using the least squares estimator. The best known risk bound, as had appeared in \citet{GSvex}, scaled like $\log(en) n^{-4/5}$ under the mean squared…
The least squares of depth-trimmed (LST) residuals regression, proposed and studied in Zuo and Zuo (2023), serves as a robust alternative to the classic least squares (LS) regression as well as a strong competitor to the renowned robust…
The bridge regression estimator generalizes both ridge regression and LASSO estimators. Since it minimizes the sum of squared residuals with a $L_{\gamma }$ penalty, this estimator is typically not robust against outliers in the data. There…
We provide a unified analysis of the predictive risk of ridge regression and regularized discriminant analysis in a dense random effects model. We work in a high-dimensional asymptotic regime where $p, n \to \infty$ and $p/n \to \gamma \in…
Multiplicative errors in addition to spatially referenced observations often arise in geodetic applications, particularly in surface estimation with light detection and ranging (LiDAR) measurements. However, spatial regression involving…
We introduce an original method of multidimensional ridge penalization in functional local linear regressions. The nonparametric regression of functional data is extended from its multivariate counterpart, and is known to be sensitive to…
The linear coefficient in a partially linear model with confounding variables can be estimated using double machine learning (DML). However, this DML estimator has a two-stage least squares (TSLS) interpretation and may produce overly wide…
We consider the problem of robustifying high-dimensional structured estimation. Robust techniques are key in real-world applications which often involve outliers and data corruption. We focus on trimmed versions of structurally regularized…
In this paper, we study nonparametric estimation of instrumental variable (IV) regressions. Recently, many flexible machine learning methods have been developed for instrumental variable estimation. However, these methods have at least one…
We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find…
We analyze the prediction error of ridge regression in an asymptotic regime where the sample size and dimension go to infinity at a proportional rate. In particular, we consider the role played by the structure of the true regression…
Iteratively Re-weighted Least Squares (IRLS) is a method for solving minimization problems involving non-quadratic cost functions, perhaps non-convex and non-smooth, which however can be described as the infimum over a family of quadratic…
Supervised learning by extreme learning machines resp. neural networks with random weights is studied under a non-stationary spatial-temporal sampling design which especially addresses settings where an autonomous object moving in a…
We propose a natural intrinsic extension of ridge regression from Euclidean spaces to general Riemannian manifolds for time-series prediction. Our approach combines Riemannian least-squares fitting via B\'ezier curves, empirical covariance…
We propose an adaptive ridge (AR) estimation scheme for a heteroscedastic linear regression model with log-linear noise in data. We simultaneously estimate the mean and variance parameters, demonstrating new asymptotic distributional and…
From benign overfitting in overparameterized models to rich power-law scalings in performance, simple ridge regression displays surprising behaviors sometimes thought to be limited to deep neural networks. This balance of phenomenological…
This paper studies the challenging problem of estimating causal effects from observational data, in the presence of unobserved confounders. The two-stage least square (TSLS) method and its variants with a standard instrumental variable (IV)…