Related papers: A Tutorial on Linear Least Square Estimation
Least squares (LS) fitting is one of the most fundamental techniques in science and engineering. It is used to estimate parameters from multiple noisy observations. In many problems the parameters are known a-priori to be bounded integer…
We prove that the ordinary least-squares (OLS) estimator attains nearly minimax optimal performance for the identification of linear dynamical systems from a single observed trajectory. Our upper bound relies on a generalization of…
In this paper, we address the problem of parameter estimation of a 2-D chirp model under the assumption that the errors are stationary. We extend the 2-D periodogram method for the sinusoidal model, to find initial values to use in any…
We study filtering of multiscale dynamical systems with model error arising from unresolved smaller scale processes. The analysis assumes continuous-time noisy observations of all components of the slow variables alone. For a linear model…
In this paper, we give a tutorial on asymptotic properties of the Least Square (LS) and Regularized Least Squares (RLS) estimators for the finite impulse response model with filtered white noise inputs. We provide three perspectives: the…
Broadband signal transmission over frequency-selective fading channel often requires accurate channel state information at receiver. One of the most attracting adaptive channel estimation methods is least mean square (LMS) algorithm.…
Small area estimation has received enormous attention in recent years due to its wide range of application, particularly in policy making decisions. The variance based on direct sample size of small area estimator is unduly large and there…
This paper develops a class of Bayesian non- and semiparametric methods for estimating regression curves and surfaces. The main idea is to model the regression as locally linear, and then place suitable local priors on the local parameters.…
This paper studies the subspace segmentation problem which aims to segment data drawn from a union of multiple linear subspaces. Recent works by using sparse representation, low rank representation and their extensions attract much…
Under distribution uncertainty, on the basis of discrete data we investigate the consistency of the least squares estimator (LSE) of the parameter for the stochastic differential equation (SDE) where the noise are characterized by…
Least-squares approximation is one of the most important methods for recovering an unknown function from data. While in many applications the data is fixed, in many others there is substantial freedom to choose where to sample. In this…
The method of quasi-optimal weights provides a comprehensive, asymptotically optimal, transparent and flexible alternative to the least squares method. The optimality holds for a general non-linear, non-gaussian case.
The Gauss Markov theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via…
This note examines the behavior of generalization capabilities - as defined by out-of-sample mean squared error (MSE) - of Linear Gaussian (with a fixed design matrix) and Linear Least Squares regression. Particularly, we consider a…
Linear regression is a basic and widely-used methodology in data analysis. It is known that some quantum algorithms efficiently perform least squares linear regression of an exponentially large data set. However, if we obtain values of the…
Nonlinear function estimation is core to modern machine learning applications. In this paper, to perform nonlinear function estimation, we reduce a nonlinear inverse problem to a linear one using a polynomial kernel expansion. These kernels…
The approximation of tensors has important applications in various disciplines, but it remains an extremely challenging task. It is well known that tensors of higher order can fail to have best low-rank approximations, but with an important…
We develop a Levenberg-Marquardt method for minimizing the sum of a smooth nonlinear least-squar es term $f(x) = \tfrac{1}{2} \|F(x)\|_2^2$ and a nonsmooth term $h$. Both $f$ and $h$ may be nonconvex. Steps are computed by minimizing the…
The least squares of depth trimmed (LST) residuals regression, proposed in Zuo and Zuo (2023) \cite{ZZ23}, serves as a robust alternative to the classic least squares (LS) regression as well as a strong competitor to the famous least…
As quantum tomography is becoming a key component of the quantum engineering toolbox, there is a need for a deeper understanding of the multitude of estimation methods available. Here we investigate and compare several such methods: maximum…