Related papers: Spectral Universality of Regularized Linear Regres…
We consider the dynamic linear regression problem, where the predictor vector may vary with time. This problem can be modeled as a linear dynamical system, with non-constant observation operator, where the parameters that need to be learned…
Over the last decade, a wave of research has characterized the exact asymptotic risk of many high-dimensional models in the proportional regime. Two foundational results have driven this progress: Gaussian universality, which shows that the…
New local linear estimators are proposed for a wide class of nonparametric regression models. The estimators are uniformly consistent regardless of satisfying traditional conditions of depen\-dence of design elements. The estimators are the…
In this paper, we study the issue of estimating a structured signal $x_0 \in \mathbb{R}^n$ from non-linear and noisy Gaussian observations. Supposing that $x_0$ is contained in a certain convex subset $K \subset \mathbb{R}^n$, we prove that…
The parallel alternating direction method of multipliers (ADMM) algorithm is widely recognized for its effectiveness in handling large-scale datasets stored in a distributed manner, making it a popular choice for solving statistical…
The spectra of random feature matrices provide essential information on the conditioning of the linear system used in random feature regression problems and are thus connected to the consistency and generalization of random feature models.…
In this paper, we introduce the \textbf{G}eneralized \textbf{L}inear \textbf{S}pectral \textbf{S}tatistics (GLSS) of a high-dimensional sample covariance matrix $\bm{S}_n$, denoted as $\operatorname{tr}f(\bm{S}_n)\bm{B}_n$, which…
We analyze gradient descent with randomly weighted data points in a linear regression model, under a generic weighting distribution. This includes various forms of stochastic gradient descent, importance sampling, but also extends to…
In this paper we consider regression problems subject to arbitrary noise in the operator or design matrix. This characterization appropriately models many physical phenomena with uncertainty in the regressors. Although the problem has been…
Diffusion models trained on different, non-overlapping subsets of a dataset often produce strikingly similar outputs when given the same noise seed. We trace this consistency to a simple linear effect: the shared Gaussian statistics across…
The tensorial principal component analysis is a generalization of ordinary principal component analysis, focusing on data which are suitably described by tensors rather than matrices. This paper aims at giving the nonperturbative…
Consider the problem of estimating the entries of a large matrix, when the observed entries are noisy versions of a small random fraction of the original entries. This problem has received widespread attention in recent times, especially…
Finding an unconstrained and statistically interpretable reparameterization of a covariance matrix is still an open problem in statistics. Its solution is of central importance in covariance estimation, particularly in the recent…
Identification of time-varying linear systems, which introduce both time-shifts (delays) and frequency-shifts (Doppler-shifts), is a central task in many engineering applications. This paper studies the problem of identification of…
We introduce a new consistency-based approach for defining and solving nonnegative/positive matrix and tensor completion problems. The novelty of the framework is that instead of artificially making the problem well-posed in the form of an…
Generalized linear models are flexible tools for the analysis of diverse datasets, but the classical formulation requires that the parametric component is correctly specified and the data contain no atypical observations. To address these…
Consider the case that we observe $n$ independent and identically distributed copies of a random variable with a probability distribution known to be an element of a specified statistical model. We are interested in estimating an infinite…
We study high-dimensional least-squares regression within a subgaussian statistical learning framework with heterogeneous noise. It includes $s$-sparse and $r$-low-rank least-squares regression when a fraction $\epsilon$ of the labels are…
In Selk and Gertheiss (2022) a nonparametric prediction method for models with multiple functional and categorical covariates is introduced. The dependent variable can be categorical (binary or multi-class) or continuous, thus both…
We propose a new randomized optimization method for high-dimensional problems which can be seen as a generalization of coordinate descent to random subspaces. We show that an adaptive sampling strategy for the random subspace significantly…