Related papers: Differentially Private Ordinary Least Squares
Random forest (RF) is one of the most popular methods for estimating regression functions. The local nature of the RF algorithm, based on intra-node means and variances, is ideal when errors are i.i.d. For dependent error processes like…
Estimation and inference in statistics pose significant challenges when data are collected adaptively. Even in linear models, the Ordinary Least Squares (OLS) estimator may fail to exhibit asymptotic normality for single coordinate…
Orthogonal least squares (OLS) is a classic algorithm for sparse recovery, function approximation, and subset selection. In this paper, we analyze the performance guarantee of the OLS algorithm. Specifically, we show that OLS guarantees the…
We provide computationally efficient, differentially private algorithms for the classical regression settings of Least Squares Fitting, Binary Regression and Linear Regression with unbounded covariates. Prior to our work, privacy…
We study the problem of identification of linear dynamical system from a single trajectory, via excitations of isotropic Gaussian. In stark contrast with previously reported results, Ordinary Least Squares (OLS) estimator for even…
This paper studies the properties of linear regression on centrality measures when network data is sparse and observed with error. We make three contributions in this setting. First, we show that OLS estimators can become inconsistent under…
The log transformation is widely used in linear regression, mainly because coefficients are interpretable as proportional effects. Yet this practice has fundamental limitations, most notably that the log is undefined at zero, creating an…
This paper proposes a method for estimating multiple change points in panel data models with unobserved individual effects via ordinary least-squares (OLS). Typically, in this setting, the OLS slope estimators are inconsistent due to the…
We show, using three empirical applications, that linear regression estimates predicated on the assumption of sparsity are fragile in two ways. First, we document that different choices of the regressor matrix which do not impact ordinary…
Uncertainty in estimating the log-law parameters is arguably the greatest obstacle to establishing definitive conclusions regarding their numerical values and universality. This challenge is exacerbated by the limited number of studies that…
Linear regression using ordinary least squares (OLS) is a critical part of every statistician's toolkit. In R, this is elegantly implemented via lm() and its related functions. However, the statistical inference output from this suite of…
The method of ``Total Least Squares'' is proposed as a more natural way (than ordinary least squares) to approximate the data if both the matrix and and the right-hand side are contaminated by ``errors''. In this tutorial note, we give a…
Latest least squares regression (LSR) methods mainly try to learn slack regression targets to replace strict zero-one labels. However, the difference of intra-class targets can also be highlighted when enlarging the distance between…
Motivated by questions about dense (non-sparse) signals in high-dimensional data analysis, we study the unconditional out-of-sample prediction error (predictive risk) associated with three popular linear estimators for high-dimensional…
This book is meant to provide an introduction to linear models and the theories behind them. Our goal is to give a rigorous introduction to the readers with prior exposure to ordinary least squares. In machine learning, the output is…
Scaled sparse linear regression jointly estimates the regression coefficients and noise level in a linear model. It chooses an equilibrium with a sparse regression method by iteratively estimating the noise level via the mean residual…
This paper introduces and analyzes a framework that accommodates general heterogeneity in regression modeling. It demonstrates that regression models with fixed or time-varying parameters can be estimated using the OLS and time-varying OLS…
Iteratively reweighted least squares (IRLS) is a widely-used method in machine learning to estimate the parameters in the generalised linear models. In particular, IRLS for L1 minimisation under the linear model provides a closed-form…
For multiple treatments D=0,1,...,J, covariates X and outcome Y, the ordinary least squares estimator (OLS) of Y on (D1,...,DJ,X) is widely applied to a constant-effect linear model, where Dj is the dummy variable for D=j. However, the…
Least squares fitting is in general not useful for high-dimensional linear models, in which the number of predictors is of the same or even larger order of magnitude than the number of samples. Theory developed in recent years has coined a…