Related papers: Conjugate Direction Methods Under Inconsistent Sys…
The efficient solution of large-scale multiterm linear matrix equations is a challenging task in numerical linear algebra, and it is a largely open problem. We propose a new iterative scheme for symmetric and positive definite operators,…
Bi-conjugate gradient (Bi-CG) and bi-conjugate residual (Bi-CR) methods are underlying iterative solvers for linear systems with nonsymmetric matrices. Residual smoothing is a standard technique for obtaining smooth convergence behavior of…
Gradient Descent (GD) and Conjugate Gradient (CG) methods are among the most effective iterative algorithms for solving unconstrained optimization problems, particularly in machine learning and statistical modeling, where they are employed…
We give a derivation of the method of conjugate gradients based on the requirement that each iterate minimizes a strictly convex quadratic on the space spanned by the previously observed gradients. Rather than verifying that the search…
The state-of-the-art methods for solving optimization problems in big dimensions are variants of randomized coordinate descent (RCD). In this paper we introduce a fundamentally new type of acceleration strategy for RCD based on the…
The residual cutting (RC) method has been proposed as an outer-inner loop iteration for efficiently solving large and sparse linear systems of equations arising in solving numerically problems of elliptic partial differential equations.…
A simple alternative to the conjugate gradient(CG) method is presented; this method is developed as a special case of the more general iterated Ritz method (IRM) for solving a system of linear equations. This novel algorithm is not based on…
We present a novel deep learning approach to approximate the solution of large, sparse, symmetric, positive-definite linear systems of equations. These systems arise from many problems in applied science, e.g., in numerical methods for…
The preconditioned conjugate gradient (PCG) algorithm is one of the most popular algorithms for solving large-scale linear systems Ax = b, where A is a symmetric positive definite matrix. Rather than computing residuals directly, it updates…
Conjugate Gradient (CG) methods are one of the most effective iterative methods to solve linear equations in Hilbert spaces. So far, they have been inherently bound to these spaces since they make use of the inner product structure. In more…
A recent article introduced thecontinuous stochastic gradient method (CSG) for the efficient solution of a class of stochastic optimization problems. While the applicability of known stochastic gradient type methods is typically limited to…
The adjoint method is an efficient way to numerically compute gradients in optimization problems with constraints, but is only formulated to differentiable cost and constraint functions on real variables. With the introduction of complex…
In this paper, we present a conditional gradient type (CGT) method for solving a class of composite optimization problems where the objective function consists of a (weakly) smooth term and a (strongly) convex regularization term. While…
This paper proposes distributed adaptive algorithms based on the conjugate gradient (CG) method and the diffusion strategy for parameter estimation over sensor networks. We present sparsity-aware conventional and modified distributed CG…
In this contribution, we present a numerical analysis of the continuous stochastic gradient (CSG) method, including applications from topology optimization and convergence rates. In contrast to standard stochastic gradient optimization…
This paper presents a general description of a parameter estimation inverse problem for systems governed by nonlinear differential equations. The inverse problem is presented using optimal control tools with state constraints, where the…
Estimation of actual errors from the residue in iterative solutions is necessary for efficient solution of large problems when their condition number is much larger than one. Such estimators for conjugate gradient algorithms used to solve…
We present an iterative method to diagonalise large matrices. The basic idea is the same as the conjugated gradient (CG) method, i.e, minimizing the Rayleigh quotient via its gradient and avoiding reintroduce errors to the directions of…
Lattice QCD calculations require significant computational effort, with the dominant fraction of resources typically spent in the numerical inversion of the Dirac operator. One of the simplest methods to solve such large and sparse linear…
We consider standard gradient descent, gradient flow and conjugate gradients as iterative algorithms for minimising a penalised ridge criterion in linear regression. While it is well known that conjugate gradients exhibit fast numerical…