Related papers: Nonparametric Least Squares Estimation of a Multiv…

We study the least squares estimator in the residual variance estimation context. We show that the mean squared differences of paired observations are asymptotically normally distributed. We further establish that, by regressing the mean…

An abstract indefinite least squares problem with a quadratic constraint is considered. This is a quadratic programming problem with one quadratic equality constraint, where neither the objective nor the constraint are convex functions.…

Estimating linear, mean-square continuous functionals is a pivotal challenge in statistics. In high-dimensional contexts, this estimation is often performed under the assumption of exact model sparsity, meaning that only a small number of…

This paper presents a model selection technique of estimation in semiparametric regression models of the type Y_i=\beta^{\prime}\underbarX_i+f(T_i)+W_i, i=1,...,n. The parametric and nonparametric components are estimated simultaneously by…

Multivariate linear regression models often face the problem of heteroscedasticity caused by multiple explanatory variables. The weighted least squares estimation with univariate-dependent weights has limitations in constructing weight…

Least squares linear regression is one of the oldest and widely used data analysis tools. Although the theoretical analysis of the ordinary least squares (OLS) estimator is as old, several fundamental questions are yet to be answered.…

Many regularization schemes for high-dimensional regression have been put forward. Most require the choice of a tuning parameter, using model selection criteria or cross-validation schemes. We show that a simple non-negative or…

Tensor regression has attracted significant attention in statistical research. This study tackles the challenge of handling covariates with smooth varying structures. We introduce a novel framework, termed functional tensor regression,…

We propose a formulation for nonlinear recurrent models that includes simple parametric models of recurrent neural networks as a special case. The proposed formulation leads to a natural estimator in the form of a convex program. We provide…

The problem of fitting experimental data to a given model function $f(t; p_1,p_2,\dots,p_N)$ is conventionally solved numerically by methods such as that of Levenberg-Marquardt, which are based on approximating the Chi-squared measure of…

We develop a technique for establishing lower bounds on the sample complexity of Least Squares (or, Empirical Risk Minimization) for large classes of functions. As an application, we settle an open problem regarding optimality of Least…

We consider a general nonparametric regression model called the compound model. It includes, as special cases, sparse additive regression and nonparametric (or linear) regression with many covariates but possibly a small number of relevant…

The aim of this paper is to define a nonlinear least squares estimator for the spectral parameters of a spherical autoregressive process of order 1 in a parametric setting. Furthermore, we investigate on its asymptotic properties, such as…

We study the monotone single index model where a real response variable $Y $ is linked to a $d$-dimensional covariate $X$ through the relationship $E[Y | X] = \Psi_0(\alpha^T_0 X)$ almost surely. Both the ridge function, $\Psi_0$, and the…

We study a minimax risk of estimating inverse functions on a plane, while keeping an estimator is also invertible. Learning invertibility from data and exploiting an invertible estimator are used in many domains, such as statistics,…

We present a formula for the shrinkage factors of the Partial Least Squares regression estimator and deduce some of their properties, in particular the known fact that some of the factors are >1. We investigate the effect of shrinkage…

We study the problem of variable selection in convex nonparametric regression. Under the assumption that the true regression function is convex and sparse, we develop a screening procedure to select a subset of variables that contains the…

Under distribution uncertainty, on the basis of discrete data we investigate the consistency of the least squares estimator (LSE) of the parameter for the stochastic differential equation (SDE) where the noise are characterized by…

In this work we are interested in the problems of supervised learning and variable selection when the input-output dependence is described by a nonlinear function depending on a few variables. Our goal is to consider a sparse nonparametric…

Estimating linear regression using least squares and reporting robust standard errors is very common in financial economics, and indeed, much of the social sciences and elsewhere. For thick tailed predictors under heteroskedasticity this…