Related papers: Randomized batch-sampling Kaczmarz methods for sol…
An algorithmic framework to compute sparse or minimal-TV solutions of linear systems is proposed. The framework includes both the Kaczmarz method and the linearized Bregman method as special cases and also several new methods such as a…
In many applications, data is collected in batches, some of which are corrupt or even adversarial. Recent work derived optimal robust algorithms for estimating discrete distributions in this setting. We consider a general framework of…
We study problem-dependent rates, i.e., generalization errors that scale near-optimally with the variance, the effective loss, or the gradient norms evaluated at the "best hypothesis." We introduce a principled framework dubbed "uniform…
In this paper, we propose a unified convergence analysis for a class of generic shuffling-type gradient methods for solving finite-sum optimization problems. Our analysis works with any sampling without replacement strategy and covers many…
A randomized subspace action algorithm is investigated for fusion frame signal recovery problems. It is noted that Kaczmarz bounds provide upper bounds on the algorithm's error moments. The main question of which probability distributions…
Probabilistic ideas and tools have recently begun to permeate into several fields where they had traditionally not played a major role, including fields such as numerical linear algebra and optimization. One of the key ways in which these…
We propose a new randomized method for solving systems of nonlinear equations, which can find sparse solutions or solutions under certain simple constraints. The scheme only takes gradients of component functions and uses Bregman…
Convergence analysis of block iterative solvers for Hermitian eigenvalue problems and the closely related research on properties of matrix-based signal filters are challenging, and attract increasing attention due to their recent…
Recent results in one-bit sampling provide a framework for a relatively low-cost, low-power sampling, at a high rate by employing time-varying sampling threshold sequences. Another recent development in sampling theory is unlimited…
Markov chain Monte Carlo (MCMC) sampling is an important and commonly used tool for the analysis of hierarchical models. Nevertheless, practitioners generally have two options for MCMC: utilize existing software that generates a black-box…
We propose a new asynchronous parallel block-descent algorithmic framework for the minimization of the sum of a smooth nonconvex function and a nonsmooth convex one, subject to both convex and nonconvex constraints. The proposed framework…
Large-scale linear systems, $Ax=b$, frequently arise in practice and demand effective iterative solvers. Often, these systems are noisy due to operational errors or faulty data-collection processes. In the past decade, the randomized…
Randomized iterative algorithms, such as the randomized Kaczmarz method and the randomized Gauss-Seidel method, have gained considerable popularity due to their efficacy in solving matrix-vector and matrix-matrix regression problems. Our…
As the number of samples and dimensionality of optimization problems related to statistics an machine learning explode, block coordinate descent algorithms have gained popularity since they reduce the original problem to several smaller…
Block coordinate descent methods and stochastic subgradient methods have been extensively studied in optimization and machine learning. By combining randomized block sampling with stochastic subgradient methods based on dual averaging, we…
Recently proposed adaptive Sketch & Project (SP) methods connect several well-known projection methods such as Randomized Kaczmarz (RK), Randomized Block Kaczmarz (RBK), Motzkin Relaxation (MR), Randomized Coordinate Descent (RCD), Capped…
The inference procedure for the mean of a stationary time series is usually quite different under various model assumptions because the partial sum process behaves differently depending on whether the time series is short or long-range…
The Sampling Kaczmarz Motzkin (SKM) algorithm is a generalized method for solving large scale linear systems of inequalities. Having its root in the relaxation method of Agmon, Schoenberg, and Motzkin and the randomized Kaczmarz method, SKM…
Random sampling has become a critical tool in solving massive matrix problems. For linear regression, a small, manageable set of data rows can be randomly selected to approximate a tall, skinny data matrix, improving processing time…
A new efficient ensemble prediction strategy is developed for a general turbulent model framework with emphasis on the nonlinear interactions between large and small scale variables. The high computational cost in running large ensemble…