Related papers: A Gauss-Seidel Iterative Thresholding Algorithm fo…
Iterative hard thresholding (IHT) is a projected gradient descent algorithm, known to achieve state of the art performance for a wide range of structured estimation problems, such as sparse inference. In this work, we consider IHT as a…
In this paper, we study the convergence properties of the Stochastic Gradient Descent (SGD) method for finding a stationary point of a given objective function $J(\cdot)$. The objective function is not required to be convex. Rather, our…
In this paper, we consider a squared $L_1/L_2$ regularized model for sparse signal recovery from noisy measurements. We first establish the existence of optimal solutions to the model under mild conditions. Next, we propose a proximal…
Instrumental variable (IV) analysis is widely used in fields such as economics and epidemiology to address unobserved confounding and measurement error when estimating the causal effects of intermediate covariates on outcomes. However,…
We propose an online learning algorithm for a class of machine learning models under a separable stochastic approximation framework. The essence of our idea lies in the observation that certain parameters in the models are easier to…
Iteratively reweighted least square (IRLS) is a popular approach to solve sparsity-enforcing regression problems in machine learning. State of the art approaches are more efficient but typically rely on specific coordinate pruning schemes.…
The Iterative Hard Thresholding (IHT) algorithm has been considered extensively as an effective deterministic algorithm for solving sparse optimizations. The IHT algorithm benefits from the information of the batch (full) gradient at each…
In this note, we consider a special instance of the scaled, inexact and adaptive generalised Fast Iterative Soft-Thresholding Algorithm (SAGE-FISTA) recently proposed in (Rebegoldi, Calatroni, '21) for the efficient solution of strongly…
A major enterprise in compressed sensing and sparse approximation is the design and analysis of computationally tractable algorithms for recovering sparse, exact or approximate, solutions of underdetermined linear systems of equations. Many…
We provide the first global model recovery results for the IRLS (iteratively reweighted least squares) heuristic for robust regression problems. IRLS is known to offer excellent performance, despite bad initializations and data corruption,…
This study presents a novel mixed-precision iterative refinement algorithm, GADI-IR, within the general alternating-direction implicit (GADI) framework, designed for efficiently solving large-scale sparse linear systems. By employing…
Recent work by Rauhut and Ward developed a notion of weighted sparsity and a corresponding notion of Restricted Isometry Property for the space of weighted sparse signals. Using these notions, we pose a best weighted sparse approximation…
We propose a randomized first order optimization algorithm Gradient Projection Iterative Sketch (GPIS) and an accelerated variant for efficiently solving large scale constrained Least Squares (LS). We provide theoretical convergence…
Quadratic regression involves modeling the response as a (generalized) linear function of not only the features $x^{j_1}$ but also of quadratic terms $x^{j_1}x^{j_2}$. The inclusion of such higher-order "interaction terms" in regression…
We develop new stochastic gradient methods for efficiently solving sparse linear regression in a partial attribute observation setting, where learners are only allowed to observe a fixed number of actively chosen attributes per example at…
The convergence of the Conjugate Gradient method is subject to a locality limitation which imposes a lower bound on the number of iterations required before a qualitatively accurate approximation can be obtained. This limitation originates…
Successive quadratic approximations (SQA) are numerically efficient for minimizing the sum of a smooth function and a convex function. The iteration complexity of inexact SQA methods has been analyzed recently. In this paper, we present an…
For high dimensional sparse linear regression problems, we propose a sequential convex relaxation algorithm (iSCRA-TL1) by solving inexactly a sequence of truncated $\ell_1$-norm regularized minimization problems, in which the working index…
We present an iterative support shrinking algorithm for $\ell_{p}$-$\ell_{q}$ minimization~($0 <p < 1 \leq q < \infty $). This algorithm guarantees the nonexpensiveness of the signal support set and can be easily implemented after being…
In statistics, generalized linear models (GLMs) are widely used for modeling data and can expressively capture potential nonlinear dependence of the model's outcomes on its covariates. Within the broad family of GLMs, those with binary…