Related papers: Random Reordering in SOR-Type Methods
While SGD, which samples from the data with replacement is widely studied in theory, a variant called Random Reshuffling (RR) is more common in practice. RR iterates through random permutations of the dataset and has been shown to converge…
Motivated by applications in single-cell biology and metagenomics, we investigate the problem of matrix reordering based on a noisy disordered monotone Toeplitz matrix model. We establish the fundamental statistical limit for this problem…
We extend results known for the randomized Gauss-Seidel and the Gauss-Southwell methods for the case of a Hermitian and positive definite matrix to certain classes of non-Hermitian matrices. We obtain convergence results for a whole range…
Randomized iterative algorithms have attracted much attention in recent years because they can approximately solve large-scale linear systems of equations without accessing the entire coefficient matrix. In this paper, we propose two novel…
Randomized iterative algorithms for solving a factorized linear system, $\mathbf A\mathbf B\mathbf x=\mathbf b$ with $\mathbf A\in{\mathbb{R}}^{m\times \ell}$, $\mathbf B\in{\mathbb{R}}^{\ell\times n}$, and $\mathbf b\in{\mathbb{R}}^m$,…
In this paper, to solve a broad class of complex symmetric linear systems, we recast the complex system in a real formulation and apply the generalized successive overrelaxation (GSOR) iterative method to the equivalent real system. We then…
In this paper we present a random shuffling scheme to apply with adaptive sorting algorithms. Adaptive sorting algorithms utilize the presortedness present in a given sequence. We have probabilistically increased the amount of presortedness…
We study a version of the randomized Kaczmarz algorithm for solving systems of linear equations where the iterates are confined to the solution space of a selected subsystem. We show that the subspace constraint leads to an accelerated…
The Kaczmarz method for solving a linear system $Ax = b$ interprets such a system as a collection of equations $\left\langle a_i, x\right\rangle = b_i$, where $a_i$ is the $i-$th row of $A$, then picks such an equation and corrects $x_{k+1}…
The randomized Kaczmarz algorithm has received considerable attention recently because of its simplicity, speed, and the ability to approximately solve large-scale linear systems of equations. In this paper we propose randomized double and…
The random reshuffling Kaczmarz (RRK) method enjoys the simplicity and efficiency in solving linear systems as a Kaczmarz-type method, whereas it also inherits the practical improvements of the stochastic gradient descent (SGD) with random…
In this paper, we further investigate and refine the subspace-constrained preconditioning technique to enhance the theoretical and numerical convergence properties of randomized iterative methods for solving linear systems. In particular,…
We develop a novel, fundamental and surprisingly simple randomized iterative method for solving consistent linear systems. Our method has six different but equivalent interpretations: sketch-and-project, constrain-and-approximate, random…
Random Reshuffling (RR) is an algorithm for minimizing finite-sum functions that utilizes iterative gradient descent steps in conjunction with data reshuffling. Often contrasted with its sibling Stochastic Gradient Descent (SGD), RR is…
Iterative sketching and sketch-and-precondition are well-established randomized algorithms for solving large-scale, over-determined linear least-squares problems. In this paper, we introduce a new perspective that interprets Iterative…
The local convergence of alternating optimization methods with overrelaxation for low-rank matrix and tensor problems is established. The analysis is based on the linearization of the method which takes the form of an SOR iteration for a…
Several recent randomized linear algebra algorithms rely upon fast dimension reduction methods. A popular choice is the Subsampled Randomized Hadamard Transform (SRHT). In this article, we address the efficacy, in the Frobenius and spectral…
The Kaczmarz method is an iterative algorithm for solving systems of linear equations Ax=b. Theoretical convergence rates for this algorithm were largely unknown until recently when work was done on a randomized version of the algorithm. It…
Perfect sorting by reversals, a problem originating in computational genomics, is the process of sorting a signed permutation to either the identity or to the reversed identity permutation, by a sequence of reversals that do not break any…
For solving linear ill-posed problems regularization methods are required when the right hand side is with some noise. In the present paper regularized solutions are obtained by implicit iteration methods in Hilbert scales. % By exploiting…