Related papers: Minimal error momentum Bregman-Kaczmarz
The Kaczmarz algorithm is an iterative technique designed to solve consistent linear systems of equations. It falls within the category of row-action methods, focusing on handling one equation per iteration. This characteristic makes it…
We combine two iterative algorithms for solving large-scale systems of linear inequalities, the relaxation method of Agmon, Motzkin et al. and the randomized Kaczmarz method. In doing so, we obtain a family of algorithms that generalize and…
In this paper, we propose and analyze an accelerated linearized Bregman (ALB) method for solving the basis pursuit and related sparse optimization problems. This accelerated algorithm is based on the fact that the linearized Bregman (LB)…
In this paper, we consider constrained optimization problems with convex, smooth objective and constraints. We propose a new stochastic gradient algorithm, called the Stochastic Moving Ball Approximation (SMBA) method, to solve this class…
Randomized regularized Kaczmarz algorithms have recently been proposed to solve tensor recovery models with {\it consistent} linear measurements. In this work, we propose a novel algorithm based on the randomized extended Kaczmarz algorithm…
Randomized Kaczmarz (RK), Motzkin Method (MM) and Sampling Kaczmarz Motzkin (SKM) algorithms are commonly used iterative techniques for solving a system of linear inequalities (i.e., $Ax \leq b$). As linear systems of equations represent a…
In this article, we establish a class of new accelerated modulus-based iteration methods for solving the linear complementarity problem. When the system matrix is an $H_+$-matrix, we present appropriate criteria for the convergence…
We propose a computational framework named iterative local adaptive majorize-minimization (I-LAMM) to simultaneously control algorithmic complexity and statistical error when fitting high dimensional models. I-LAMM is a two-stage…
The Kaczmarz method is successfully used for solving discretizations of linear inverse problems, especially in computed tomography where it is known as ART. Practitioners often observe and appreciate its fast convergence in the first few…
Accelerating stochastic gradient methods with classical momentum schemes, such as Polyak's heavy ball, has proven highly successful in training large-scale machine learning models, particularly when combined with the hardware acceleration…
In this paper, we study an algorithm for solving a class of nonconvex and nonsmooth nonseparable optimization problems. Based on proximal alternating linearized minimization (PALM), we propose a new iterative algorithm which combines…
The stochastic heavy ball momentum (SHBM) method has gained considerable popularity as a scalable approach for solving large-scale optimization problems. However, one limitation of this method is its reliance on prior knowledge of certain…
Block Coordinate Update (BCU) methods enjoy low per-update computational complexity because every time only one or a few block variables would need to be updated among possibly a large number of blocks. They are also easily parallelized and…
We present a randomized iterative algorithm that exponentially converges in expectation to the minimum Euclidean norm least squares solution of a given linear system of equations. The expected number of arithmetic operations required to…
In this paper we develop a stochastic heavy ball method for solving ill-posed inverse problems. The method updates the iterate using only a randomly selected equation at each iteration step while incorporating a momentum term into the…
In this article we develop and analyze novel iterative regularization techniques for the solution of systems of nonlinear ill--posed operator equations. The basic idea consists in considering separately each equation of this system and…
Stochastic iterative methods are useful in a variety of large-scale numerical linear algebraic, machine learning, and statistical problems, in part due to their low-memory footprint. They are frequently used in a variety of applications,…
In this note we reconsider two known algorithms which both usually converge faster than the randomized Kaczmarz method introduced by Strohmer and Vershynin(2009), but require the additional computation of all residuals of an iteration at…
In this paper, several Kaczmarz-type numerical methods for solving the matrix equation $AX=B$ and $XA=C$ are proposed, where the coefficient matrix $A$ may be full rank or rank deficient. These methods are iterative methods without matrix…
Linear regression is effective at identifying interpretable trends in a data set, but averages out potentially different effects on subgroups within data. We propose an iterative algorithm based on the randomized Kaczmarz (RK) method to…