Related papers: A Tutorial on Linear Least Square Estimation
Least squares is by far the simplest and most commonly applied computational method in many fields. In almost all applications, the least squares objective is rarely the true objective. We account for this discrepancy by parametrizing the…
Least squares form one of the most prominent classes of optimization problems, with numerous applications in scientific computing and data fitting. When such formulations aim at modeling complex systems, the optimization process must…
Gaussian mixtures are a powerful and widely used tool to model non-Gaussian estimation problems. They are able to describe measurement errors that follow arbitrary distributions and can represent ambiguity in assignment tasks like point set…
The partial least squares procedure was originally developed to estimate the slope parameter in multivariate parametric models. More recently it has gained popularity in the functional data literature. There, the partial least squares…
The wide application of estimation techniques in system analysis enable us to best determine and understand the history of system states. This paper attempts to delineate the theory behind linear and non-linear estimation with a suitable…
We present a novel iterative algorithm for approximating the linear least squares solution with low complexity. After a motivation of the algorithm we discuss the algorithm's properties including its complexity, and we present theoretical…
This tutorial serves as an introduction to recently developed non-asymptotic methods in the theory of -- mainly linear -- system identification. We emphasize tools we deem particularly useful for a range of problems in this domain, such as…
The a posteriori error estimator using the least-squares functional can be used for adaptive mesh refinement and error control even if the numerical approximations are not obtained from the corresponding least-squares method. This suggests…
We investigate the nonlinear regression problem under L2 loss (square loss) functions. Traditional nonlinear regression models often result in non-convex optimization problems with respect to the parameter set. We show that a convex…
In the famous least sum of trimmed squares (LTS) of residuals estimator (Rousseeuw (1984)), residuals are first squared and then trimmed. In this article, we first trim residuals - using a depth trimming scheme - and then square the rest of…
We study the problem of parameter estimation for reflected stochastic processes driven by a standard Brownian motion. The estimator is obtained using nonlinear least squares method based on discretely observed processes. Under some certain…
When a physical system is modeled by a nonlinear function, the unknown parameters can be estimated by fitting experimental observations by a least-squares approach. Newton's method and its variants are often used to solve problems of this…
Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an $l_0$-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm…
In this note, we concentrate on the backward error of the equality constrained indefinite least squares problem. For the normwise backward error of the equality constrained indefinite least square problem, we adopt the linearization method…
This article investigates the least squares estimators (LSE) for the unknown parameters in stochastic differential equations (SDEs) that are affected by L\'evy noise, particularly when the sample paths are sparse. Specifically, given $n$…
Nonlinear least-squares problems are a special class of unconstrained optimization problems in which their gradient and Hessian have special structures. In this paper, we exploit these structures and proposed a matrix-free algorithm with a…
We analyze a simple prefiltered variation of the least squares estimator for the problem of estimation with biased, semi-parametric noise, an error model studied more broadly in causal statistics and active learning. We prove an oracle…
A new statistical technique for constructing linear latent structure (LLS) models from available data, supported by well established theoretical results and an efficient algorithm, is presented. The method reduces the problem of estimating…
Partial least squares (PLS) is a dimensionality reduction technique used as an alternative to ordinary least squares (OLS) in situations where the data is colinear or high dimensional. Both PLS and OLS provide mean based estimates, which…
Implementing state estimation in low and medium voltage power distribution is still challenging given the scale of many networks and the reliance of traditional methods on a large number of measurements. This paper proposes a method to…