Related papers: Convergence Properties of Nonlinear GMRES Applied …
In this work, we revisit nonlinear generalized minimal residual method (NGMRES) applied to nonlinear problems. NGMRES is used to accelerate the convergence of fixed-point iterations, which can substantially improve the performance of the…
We consider nonlinear convergence acceleration methods for fixed-point iteration $x_{k+1}=q(x_k)$, including Anderson acceleration (AA), nonlinear GMRES (NGMRES), and Nesterov-type acceleration (corresponding to AA with window size one). We…
In this work, we develop an alternating nonlinear Generalized Minimum Residual (NGMRES) algorithm with depth $m$ and periodicity $p$, denoted by aNGMRES($m, p$), applied to linear systems. We provide a theoretical analysis to quantify by…
In this work, we propose new variants of Anderson acceleration and nonlinear GMRES for general fixed-point iterations, based on modified least-squares problems associated with the methods. To solve the underlying linear systems, we apply…
GMRES is a popular Krylov subspace method for solving linear systems of equations involving a general non-Hermitian coefficient matrix. The conventional bounds on GMRES convergence involve polynomial approximation problems in the complex…
This paper has proposed the GMRES that augments Krylov subspaces with a set of approximate right singular vectors. The proposed method suppresses the error norms of a linear system of equations. Numerical experiments comparing the proposed…
This paper is about GMRES algorithms for the solution of nonsingular linear systems. We first consider basic algorithms and study their convergence. We then focus on acceleration strategies and parallel algorithms that are useful for…
We consider nonlinear GMRES (NGMRES) as an acceleration technique for the Navier-Stokes Picard iteration, a direction that has not previously been explored. We identify the optimal norm for the least squares optimization problem arising in…
Nonlinear acceleration algorithms improve the performance of iterative methods, such as gradient descent, using the information contained in past iterates. However, their efficiency is still not entirely understood even in the quadratic…
In this paper, we study the restarted Krylov subspace method, which is typically represented by the GMRES(m) method. Our work mainly focused on the amount of change in the iterative solution of GMRES(m) at each restart. We propose an…
Nesterov's well-known scheme for accelerating gradient descent in convex optimization problems is adapted to accelerating stationary iterative solvers for linear systems. Compared with classical Krylov subspace acceleration methods, the…
Inverse problems arise in various scientific and engineering applications, necessitating robust numerical methods for their solution. In this work, we consider the effectiveness of Krylov subspace iterative methods, including GMRES, QMR,…
Nonlinear acceleration methods are powerful techniques to speed up fixed-point iterations. However, many acceleration methods require storing a large number of previous iterates and this can become impractical if computational resources are…
Gradient restarting has been shown to improve the numerical performance of accelerated gradient methods. This paper provides a mathematical analysis to understand these advantages. First, we establish global linear convergence guarantees…
When the CG method for solving linear algebraic systems was formulated about 70 years ago by Lanczos, Hestenes, and Stiefel, it was considered an iterative process possessing a mathematical finite termination property. CG was placed into a…
A new algorithm is presented for computing a canonical rank-R tensor approximation that has minimal distance to a given tensor in the Frobenius norm, where the canonical rank-R tensor consists of the sum of R rank-one components. Each…
A adapted tensor-structured GMRES method for the TT format is proposed and investigated. The Tensor Train (TT) approximation is a robust approach to high-dimensional problems. One class of problems is solution of a linear system. In this…
The GMRES method is used to solve sparse, non-symmetric systems of linear equations arising from many scientific applications. The solver performance within a single node is memory bound, due to the low arithmetic intensity of its…
The generalized Gearhart-Koshy acceleration is a recent exact affine search technique designed for the method of cyclic projections onto hyperplanes, i.e., the Kaczmarz method. However, its convergence properties, particularly the linear…
Modeling complex multiway relationships in large-scale networks is becoming more and more challenging in data science. The multilinear PageRank problem, arising naturally in the study of higher-order Markov chains, is a powerful framework…