Related papers: Unbiased estimators for random design regression
We study the least squares estimator in the residual variance estimation context. We show that the mean squared differences of paired observations are asymptotically normally distributed. We further establish that, by regressing the mean…
We study design-unbiased estimation of the finite-population total $\sum_{i=1}^N y_i$ when each outcome satisfies known bounds $y_i\in[a_i,b_i]$. For any sampling design with inclusion probabilities $\pi_i>0$, we prove a sharp lower bound…
Given any domain $X\subseteq \mathbb{R}^d$ and a probability measure $\rho$ on $X$, we study the problem of approximating in $L^2(X,\rho)$ a given function $u:X\to\mathbb{R}$, using its noiseless pointwise evaluations at random samples. For…
A variance reduction technique in nonparametric smoothing is proposed: at each point of estimation, form a linear combination of a preliminary estimator evaluated at nearby points with the coefficients specified so that the asymptotic bias…
Estimation of the prediction error of a linear estimation rule is difficult if the data analyst also use data to select a set of variables and construct the estimation rule using only the selected variables. In this work, we propose an…
Motivated by questions about dense (non-sparse) signals in high-dimensional data analysis, we study the unconditional out-of-sample prediction error (predictive risk) associated with three popular linear estimators for high-dimensional…
We consider the estimation of a structural function which models a non-parametric relationship between a response and an endogenous regressor given an instrument in presence of dependence in the data generating process. Assuming an…
In this work, we study a random orthogonal projection based least squares estimator for the stable solution of a multivariate nonparametric regression (MNPR) problem. More precisely, given an integer $d\geq 1$ corresponding to the dimension…
We consider stochastic optimization problems which use observed data to estimate essential characteristics of the random quantities involved. Sample average approximation (SAA) or empirical (plug-in) estimation are very popular ways to use…
Design-consistent model-assisted estimation has become the standard practice in survey sampling. However, a general theory is lacking so far, which allows one to incorporate modern machine-learning techniques that can lead to potentially…
Subsampling methods have been recently proposed to speed up least squares estimation in large scale settings. However, these algorithms are typically not robust to outliers or corruptions in the observed covariates. The concept of influence…
We consider the problem of designing experiments for the estimation of a target in regression analysis if there is uncertainty about the parametric form of the regression function. A new optimality criterion is proposed, which minimizes the…
We present a sample- and time-efficient differentially private algorithm for ordinary least squares, with error that depends linearly on the dimension and is independent of the condition number of $X^\top X$, where $X$ is the design matrix.…
Additive regression models are actively researched in the statistical field because of their usefulness in the analysis of responses determined by non-linear relationships with multivariate predictors. In this kind of statistical models,…
An important challenge in statistical analysis concerns the control of the finite sample bias of estimators. This problem is magnified in high-dimensional settings where the number of variables $p$ diverges with the sample size $n$, as well…
Least absolute deviation regression is applied using a fixed number of points for all values of the index to estimate the index and scale parameter of the stable distribution using regression methods based on the empirical characteristic…
We study the least square estimator, in the framework of simple linear regression, when the deviance term $\varepsilon$ with respect to the linear model is modeled by a uniform distribution. In particular, we give the law of this estimator,…
The Bayes linear estimator is derived by minimizing the Bayes risk with respect to the squared loss function. Non-unbiased estimators such as ordinary ridge, typical shrinkage, fractional rank, and restricted least squares estimators, as…
This book is meant to provide an introduction to linear models and the theories behind them. Our goal is to give a rigorous introduction to the readers with prior exposure to ordinary least squares. In machine learning, the output is…
We study the probabilistic sampling of a random variable, in which the variable is sampled only if it falls outside a given set, which is called the silence set. This helps us to understand optimal event-based sampling for the special case…