Related papers: A note on the hybrid steepest descent methods
In this book chapter, we briefly describe the main components that constitute the gradient descent method and its accelerated and stochastic variants. We aim at explaining these components from a mathematical point of view, including…
We propose and analyze an iterative high-order hybridized discontinuous Galerkin (iHDG) discretization for linear partial differential equations. We improve our previous work (SIAM J. Sci. Comput. Vol. 39, No. 5, pp. S782--S808) in several…
We introduce a framework to accelerate the convergence of gradient-based methods with online learning. The framework learns to scale the gradient at each iteration through an online learning algorithm and provably accelerates gradient-based…
Preconditioned gradient iterations for very large eigenvalue problems are efficient solvers with growing popularity. However, only for the simplest preconditioned eigensolver, namely the preconditioned gradient iteration (or preconditioned…
We propose and analyze a new parallel coordinate descent method---`NSync---in which at each iteration a random subset of coordinates is updated, in parallel, allowing for the subsets to be chosen non-uniformly. We derive convergence rates…
We propose an adaptive way to choose the anchoring parameters for the Halpern iteration to find a fixed point of a nonexpansive mapping in a real Hilbert space. We prove strong convergence of this adaptive Halpern iteration and obtain the…
We study the convergence of a random iterative sequence of a family of operators on infinite dimensional Hilbert spaces, inspired by the Stochastic Gradient Descent (SGD) algorithm in the case of the noiseless regression, as studied in [1].…
It is well known that as a famous type of iterative methods in numerical linear algebra, Gauss-Seidel iterative methods are convergent for linear systems with strictly or irreducibly diagonally dominant matrices, invertible $H-$matrices…
In this paper, inspired by Iemoto and Takahashi [S. Iemoto, W. Takahashi, Nonlinear Analysis 71, (2009), 2082-2089], we study the Halpern's method to approximate strongly fixed points of a nonexpansive mapping and of a nonspreading mapping.…
In this paper, we study the gradient descent-ascent method for convex-concave saddle-point problems. We derive a new non-asymptotic global convergence rate in terms of distance to the solution set by using the semidefinite programming…
We prove the weak and the strong convergence of the trajectories of the continuous gradient projection method under some mild assumptions on the objective function and the step size function. Moreover, we estimate the decay rate to…
Although it is relatively easy to apply, the gradient method often displays a disappointingly slow rate of convergence. Its convergence is specially based on the structure of the matrix of the algebraic linear system, and on the choice of…
In this paper, we introduce and study a new extragradient iterative process for finding a common element of the set of fixed points of an infinite family of nonexpansive mappings and the set of solutions of a variational inequality for an…
For solving linear ill-posed problems regularization methods are required when the right hand side is with some noise. In the present paper regularized solutions are obtained by implicit iteration methods in Hilbert scales. % By exploiting…
We introduce new multilevel methods for solving large-scale unconstrained optimization problems. Specifically, the philosophy of multilevel methods is applied to Newton-type methods that regularize the Newton sub-problem using second order…
In J. Sci. Comput., 81: 2188-2212, 2019, we considered a superconvergent hybridizable discontinuous Galerkin (HDG) method, defined on simplicial meshes, for scalar reaction diffusion equations and showed how to define an interpolatory…
We describe new, simple, recursive methods of construction for orientable sequences over an arbitrary finite alphabet, i.e. periodic sequences in which any sub-sequence of n consecutive elements occurs at most once in a period in either…
An efficient proximal-gradient-based method, called proximal extrapolated gradient method, is designed for solving monotone variational inequality in Hilbert space. The proposed method extends the acceptable range of parameters to obtain…
In this paper, we first study nonsmooth steepest descent method for nonsmooth functions defined on Hilbert space and establish the corresponding algorithm by proximal subgradients. Then, we use this algorithm to find stationary points for…
A Halpern-type relaxed inertial inexact progressive hedging algorithm (PHA) is proposed for solving multi-stage stochastic variational inequalities in general probability spaces. The subproblems in this algorithm are allowed to be…