Related papers: On Least Squares Estimation under Heteroscedastic …
Ordinary least squares (OLS) linear regression is one of the most basic statistical techniques for data analysis. In the main stream literature and the statistical education, the study of linear regression is typically restricted to the…
A multiple interval-valued linear regression model considering all the cross-relationships between the mids and spreads of the intervals has been introduced recently. A least-squares estimation of the regression parameters has been carried…
The minimum mean-squared error (MMSE) is one of the most popular criteria for Bayesian estimation. Conversely, the signal-to-noise ratio (SNR) is a typical performance criterion in communications, radar, and generally detection theory. In…
Least squares estimation, a regression technique based on minimisation of residuals, has been invaluable in bringing the best fit solutions to parameters in science and engineering. However, in dynamic environments such as in Geomatics…
We consider the minimization of composite objective functions composed of the expectation of quadratic functions and an arbitrary convex function. We study the stochastic dual averaging algorithm with a constant step-size, showing that it…
We study early-stopped mirror descent (ESMD) for high-dimensional Gaussian linear regression over arbitrary convex bodies and design matrices, where the task is to minimize the in-sample mean squared error. Our main result shows that some…
We propose an $\ell_1$-penalized estimator for high-dimensional models of Expected Shortfall (ES). The estimator is obtained as the solution to a least-squares problem for an auxiliary dependent variable, which is defined as a…
We propose a class of weighted least squares estimators for the tail index of a distribution function with a regularly varying upper tail. Our approach is based on the method developed by \cite{Holan2010} for the Parzen tail index.…
We present large sample results for partitioning-based least squares nonparametric regression, a popular method for approximating conditional expectation functions in statistics, econometrics, and machine learning. First, we obtain a…
We collect robust proposals given in the field of regression models with heteroscedastic errors. Our motivation stems from the fact that the practitioner frequently faces the confluence of two phenomena in the context of data analysis:…
We consider the problem of sequentially learning to estimate, in the mean squared error (MSE) sense, a Gaussian $K$-vector of unknown covariance by observing only $m < K$ of its entries in each round. We propose two MSE estimators, and…
We study the estimation error of constrained M-estimators, and derive explicit upper bounds on the expected estimation error determined by the Gaussian width of the constraint set. Both of the cases where the true parameter is on the…
We consider the least-squares regression problem and provide a detailed asymptotic analysis of the performance of averaged constant-step-size stochastic gradient descent (a.k.a. least-mean-squares). In the strongly-convex case, we provide…
This paper provides an alternative to penalized estimators for estimation and vari- able selection in high dimensional linear regression models with measurement error or missing covariates. We propose estimation via bias corrected least…
In this paper we propose a semi-parametric Bayesian Generalized Least Squares estimator. In a generic setting where each error is a vector, the parametric Generalized Least Square estimator maintains the assumption that each error vector…
The least trimmed squares (LTS) estimator is a renowned robust alternative to the classic least squares estimator and is popular in location, regression, machine learning, and AI literature. Many studies exist on LTS, including its…
We consider a linear regression model with a spatially correlated error term on a lattice. When estimating coefficients in the linear regression model, the generalized least squares estimator (GLSE) is used if the covariance structures are…
This paper considers the penalized least squares estimator with arbitrary convex penalty. When the observation noise is Gaussian, we show that the prediction error is a subgaussian random variable concentrated around its median. We apply…
The discovery of non-linear causal relationship under additive non-Gaussian noise models has attracted considerable attention recently because of their high flexibility. In this paper, we propose a novel causal inference algorithm called…
The stochastic gradient descent (SGD) optimization algorithm plays a central role in a series of machine learning applications. The scientific literature provides a vast amount of upper error bounds for the SGD method. Much less attention…