Related papers: Mixed Precision $s$-step Lanczos and Conjugate Gra…
On modern large-scale parallel computers, the performance of Krylov subspace iterative methods is limited by global synchronization. This has inspired the development of $s$-step Krylov subspace method variants, in which iterations are…
We consider three mathematically equivalent variants of the conjugate gradient (CG) algorithm and how they perform in finite precision arithmetic. It was shown in [{\em Behavior of slightly perturbed Lanczos and conjugate-gradient…
The adaptive $s$-step CG algorithm is a solver for sparse, symmetric positive definite linear systems designed to reduce the synchronization cost per iteration while still achieving a user-specified accuracy requirement. In this work, we…
Using lower precision in algorithms can be beneficial in terms of reducing both computation and communication costs. Motivated by this, we aim to further the state-of-the-art in developing and analyzing mixed precision variants of iterative…
Our goal in this paper is to clarify the relationship between the block Lanczos and the block conjugate gradient (BCG) algorithms. Under the full rank assumption for the block vectors, we show the one-to-one correspondence between the…
The Lanczos algorithm, introduced by Cornelius Lanczos, has been known for a long time and is widely used in computational physics. While often employed to approximate extreme eigenvalues and eigenvectores of an operator, recently interest…
The Lanczos method is one of the most powerful and fundamental techniques for solving an extremal symmetric eigenvalue problem. Convergence-based error estimates depend heavily on the eigenvalue gap. In practice, this gap is often…
Bilevel optimization, with broad applications in machine learning, has an intricate hierarchical structure. Gradient-based methods have emerged as a common approach to large-scale bilevel problems. However, the computation of the…
The growing availability and usage of low precision foating point formats has attracts many interests of developing lower or mixed precision algorithms for scientific computing problems. In this paper we investigate the possibility of…
In theory, the Lanczos algorithm generates an orthogonal basis of the corresponding Krylov subspace. However, in finite precision arithmetic, the orthogonality and linear independence of the computed Lanczos vectors is usually lost quickly.…
The low rank approximation of matrices is a crucial component in many data mining applications today. A competitive algorithm for this class of problems is the randomized block Lanczos algorithm - an amalgamation of the traditional block…
Enlarged Krylov subspace methods and their s-step versions were introduced [7] in the aim of reducing communication when solving systems of linear equations Ax = b. These enlarged CG methods consist of enlarging the Krylov subspace by a…
We present a variant of the s-step Preconditioned Conjugate Gradient (PCG) method that combines a Chebyshev-stabilized Krylov basis with a Forward Gauss-Seidel (FGS) iteration for the solution of the reduced Gram systems. In s-step…
Polynomial Krylov subspace methods are among the most widely used methods for approximating $f(A)b$, the action of a matrix function on a vector, in particular when $A$ is large and sparse. When $A$ is Hermitian positive definite, the…
A common approach to approximating quadratic forms of matrix functions is to use a quadrature rule derived from the Lanczos process, known as a Lanczos quadrature. Although symmetric quadrature rules are computationally favorable, it has…
In her seminal 1989 work, Greenbaum demonstrated that the results produced by the finite precision Lanczos algorithm after $k$ iterations can be interpreted as exact Lanczos results applied to a larger matrix, whose eigenvalues lie in small…
Multi-objective optimization is central to many engineering and machine learning applications, where multiple objectives must be optimized in balance. While multi-gradient based optimization methods combine these objectives in each step,…
We develop a gradient-like algorithm to minimize a sum of peer objective functions based on coordination through a peer interconnection network. The coordination admits two stages: the first is to constitute a gradient, possibly with…
We develop a block minimum residual (MINRES) algorithm for symmetric indefinite matrices. This version is built upon the band Lanczos method that generates one basis vector of the block Krylov subspace per iteration rather than a whole…
We propose an adaptive mixed precision and dynamically scaled preconditioned conjugate gradient algorithm (AMP-PCG). It dynamically adjusts the precision for storing vectors and computing, exploiting low precision when appropriate, while…