Related papers: On the Relation Between the Randomized Extended Ka…
We describe an asynchronous parallel variant of the randomized Kaczmarz (RK) algorithm for solving the linear system $Ax=b$. The analysis shows linear convergence and indicates that nearly linear speedup can be expected if the number of…
Randomized iterative algorithms have attracted much attention in recent years because they can approximately solve large-scale linear systems of equations without accessing the entire coefficient matrix. In this paper, we propose two novel…
The Kaczmarz method is an iterative numerical method for solving large and sparse rectangular systems of linear equations. Gearhart, Koshy and Tam have developed an acceleration technique for the Kaczmarz method that minimizes the distance…
Distributed linear algebraic equation over networks, where nodes hold a part of problem data and cooperatively solve the equation via node-to-node communications, is a basic distributed computation task receiving an increasing research…
Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking…
In this work, we shed light on the so-called Kaczmarz method for solving Linear System (LS) and Linear Feasibility (LF) problems from a optimization point of view. We introduce well-known optimization approaches such as Lagrangian penalty…
To efficiently solve large scale nonlinear systems, we propose a novel Random Greedy Fast Block Kaczmarz method. This approach integrates the strengths of random and greedy strategies while avoiding the computationally expensive…
Matrix factorization techniques compute low-rank product approximations of high dimensional data matrices and as a result, are often employed in recommender systems and collaborative filtering applications. However, many algorithms for this…
In this paper we propose a parallel coordinate descent algorithm for solving smooth convex optimization problems with separable constraints that may arise e.g. in distributed model predictive control (MPC) for linear network systems. Our…
In this paper, we consider low-rank approximations for the solutions to the stochastic Helmholtz equation with random coefficients. A Stochastic Galerkin finite element method is used for the discretization of the Helmholtz problem.…
Coordinate descent algorithms are popular for huge-scale optimization problems due to their low cost per-iteration. Coordinate descent methods apply to problems where the constraint set is separable across coordinates. In this paper, we…
We provide an exact analysis of a class of randomized algorithms for solving overdetermined least-squares problems. We consider first-order methods, where the gradients are pre-conditioned by an approximation of the Hessian, based on a…
Among recent developments centered around Randomized Kaczmarz (RK), a row-sampling iterative projection method for large-scale linear systems, several adaptions to the method have inspired faster convergence. Focusing solely on…
The Kaczmarz method for solving a linear system $Ax = b$ interprets such a system as a collection of equations $\left\langle a_i, x\right\rangle = b_i$, where $a_i$ is the $i-$th row of $A$, then picks such an equation and corrects $x_{k+1}…
In this paper, we consider a novel two-dimensional randomized Kaczmarz method and its improved version with simple random sampling, which chooses two active rows with probability proportional to the square of their cross-product-like…
We comment on two randomized algorithms for constructing low-rank matrix decompositions. Both algorithms employ the Subsampled Randomized Hadamard Transform [14]. The first algorithm appeared recently in [9]; here, we provide a novel…
We introduce a new iterative regularization method for solving inverse problems that can be written as systems of linear or non-linear equations in Hilbert spaces. The proposed averaged Kaczmarz (AVEK) method can be seen as a hybrid method…
We revisit and adapt the extended sequential quadratic method (ESQM) in [3] for solving a class of difference-of-convex optimization problems whose constraints are defined as the intersection of level sets of Lipschitz differentiable…
We study how the learning rate affects the performance of a relaxed randomized Kaczmarz algorithm for solving $A x \approx b + \varepsilon$, where $A x =b$ is a consistent linear system and $\varepsilon$ has independent mean zero random…
The Contrastive Divergence (CD) algorithm has achieved notable success in training energy-based models including Restricted Boltzmann Machines and played a key role in the emergence of deep learning. The idea of this algorithm is to…