Related papers: Curious Pivot Points of Least-Squares Regression
Three methods of least squares are examined for fitting a line to points in the plane. Two well known methods are to minimize sums of squares of vertical or horizontal distances to the line. Less known is to minimize sums of squares of…
Simplicial-simplicial regression refers to the regression setting where both the responses and predictor variables lie within the simplex space, i.e. they are compositional. For this setting, constrained least squares, where the regression…
Recurrence plots exhibit line structures which represent typical behaviour of the investigated system. The local slope of these line structures is connected with a specific transformation of the time scales of different segments of the…
When a linear model is adjusted to control for additional explanatory variables the sign of a fitted coefficient may reverse. Here these reversals are studied using coefficients of determination. The resulting theory can be used to…
Least-squares refitting is widely used in high dimensional regression to reduce the prediction bias of l1-penalized estimators (e.g., Lasso and Square-Root Lasso). We present theoretical and numerical results that provide new insights into…
The least squares fit to a straight line, when both variables are affected by all equal uncorrelated errors, leads to very simple results for both the estimated parameters and their standard errors, of widespread applicability. In this…
Linear least squares regression is subject to bias due to an omitted variable, a mismeasured regressor, or simultaneity. A simple test to detect the bias is proposed and explored in simulation and in real data sets.
It is shown that the the popular least squares method of option pricing converges even under very general assumptions. This substantially increases the freedom of creating different implementations of the method, with varying levels of…
We propose a recursive least-squares method with multiple forgetting schemes to track time-varying model parameters which change with different rates. Our approach hinges on the reformulation of the classic recursive least-squares with…
Least squares linear regression is one of the oldest and widely used data analysis tools. Although the theoretical analysis of the ordinary least squares (OLS) estimator is as old, several fundamental questions are yet to be answered.…
Least squares is by far the simplest and most commonly applied computational method in many fields. In almost all applications, the least squares objective is rarely the true objective. We account for this discrepancy by parametrizing the…
We describe the limit (for two topologies) of large uniform random square permutations, i.e., permutations where every point is a record. The starting point for all our results is a sampling procedure for asymptotically uniform square…
The least squares problem is formulated in terms of Lp quasi-norm regularization (0<p<1). Two formulations are considered: (i) an Lp-constrained optimization and (ii) an Lp-penalized (unconstrained) optimization. Due to the nonconvexity of…
Within the context of recursive least squares (RLS) parameter estimation, the goal of the present paper is to study the effect of regularization-induced bias on the transient and asymptotic accuracy of the parameter estimates. We consider…
Consider linear regression where the examples are generated by an unknown distribution on $R^d\times R$. Without any assumptions on the noise, the linear least squares solution for any i.i.d. sample will typically be biased w.r.t. the least…
We go through the many considerations involved in fitting a model to data, using as an example the fit of a straight line to a set of points in a two-dimensional plane. Standard weighted least-squares fitting is only appropriate when there…
We consider a finite mixture model with varying mixing probabilities. Linear regression models are assumed for observed variables with coefficients depending on the mixture component the observed subject belongs to. A modification of the…
The relationship between the number of training data points, the number of parameters, and the generalization capabilities of models has been widely studied. Previous work has shown that double descent can occur in the over-parameterized…
We prove the statistical consistency of kernel Partial Least Squares Regression applied to a bounded regression learning problem on a reproducing kernel Hilbert space. Partial Least Squares stands out of well-known classical approaches as…
In this article we study a small random perturbation of a linear recurrence equation. If all the roots of its corresponding characteristic equation have modulus strictly less than one, the random linear recurrence goes exponentially fast to…