Related papers: Statistical Limits of Adaptive Linear Models: Low-…
In this paper we consider the issue of reliability of measurements in distributed adaptive estimation problem. To this aim, we assume a sensor network with different observation noise variance among the sensors and propose new estimation…
The problem of adaptive multivariate function estimation in the single-index regression model with random design and weak assumptions on the noise is investigated. A novel estimation procedure that adapts simultaneously to the unknown index…
A significant hurdle for analyzing large sample data is the lack of effective statistical computing and inference methods. An emerging powerful approach for analyzing large sample data is subsampling, by which one takes a random subsample…
Consider the case that we observe $n$ independent and identically distributed copies of a random variable with a probability distribution known to be an element of a specified statistical model. We are interested in estimating an infinite…
Linear regression is one of the most prevalent techniques in machine learning, however, it is also common to use linear regression for its \emph{explanatory} capabilities rather than label prediction. Ordinary Least Squares (OLS) is often…
Misspecified models often provide useful information about the true data generating distribution. For example, if $y$ is a non-linear function of $x$ the least squares estimator $\hat{\beta}$ is an estimate of $\beta$, the slope of the best…
System identification is a fundamental problem in control and learning, particularly in high-stakes applications where data efficiency is critical. Classical approaches, such as the ordinary least squares estimator (OLS), achieve an…
This paper deals with subspace estimation in the small sample size regime, where the number of samples is comparable in magnitude with the observation dimension. The traditional estimators, mostly based on the sample correlation matrix, are…
We consider the adaptive Lasso estimator with componentwise tuning in the framework of a low-dimensional linear regression model. In our setting, at least one of the components is penalized at the rate of consistent model selection and…
We study the statistical properties of the least squares estimator in unimodal sequence estimation. Although closely related to isotonic regression, unimodal regression has not been as extensively studied. We show that the unimodal least…
Recent years have witnessed the success of adaptive (or unified) approaches in estimating symmetric properties of discrete distributions, where one first obtains a distribution estimator independent of the target property, and then plugs…
The log transformation is widely used in linear regression, mainly because coefficients are interpretable as proportional effects. Yet this practice has fundamental limitations, most notably that the log is undefined at zero, creating an…
We propose an algorithm to select parameter subset combinations that can be estimated using an ordinary least-squares (OLS) inverse problem formulation with a given data set. First, the algorithm selects the parameter combinations that…
Adaptive importance samplers are adaptive Monte Carlo algorithms to estimate expectations with respect to some target distribution which \textit{adapt} themselves to obtain better estimators over a sequence of iterations. Although it is…
Several classical adaptive optimization algorithms, such as line search and trust region methods, have been recently extended to stochastic settings where function values, gradients, and Hessians in some cases, are estimated via stochastic…
We derive new theoretical results on the properties of the adaptive least absolute shrinkage and selection operator (adaptive lasso) for time series regression models. In particular, we investigate the question of how to conduct finite…
This paper studies the asymptotic properties of the adaptive elastic net in ultra-high dimensional sparse linear regression models and proposes a new method called SSLS (Separate Selection from Least Squares) to improve prediction accuracy.…
We study the problem of identification of linear dynamical system from a single trajectory, via excitations of isotropic Gaussian. In stark contrast with previously reported results, Ordinary Least Squares (OLS) estimator for even…
In this article we study post-model selection estimators that apply ordinary least squares (OLS) to the model selected by first-step penalized estimators, typically Lasso. It is well known that Lasso can estimate the nonparametric…
The statistical essence of the Transformer architecture has long remained elusive: Is it a universal approximator, or a neural network version of known computational algorithms? Through rigorous algebraic proof, we show that the latter…