Related papers: Convergence Properties of Nonlinear GMRES Applied …
Alternating projection method has been used in a wide range of engineering applications since it is a gradient-free method (without requiring tuning the step size) and usually has fast speed of convergence. In this paper, we formalize two…
Nesterov SGD is widely used for training modern neural networks and other machine learning models. Yet, its advantages over SGD have not been theoretically clarified. Indeed, as we show in our paper, both theoretically and empirically,…
In this paper, we present an accurate and numerically efficient method to implement the GN and EGN nonlinearity prediction methods when the power evolution along the fiber is in an arbitrary form. This approach will provide us with a…
This paper proposes and develops new linesearch methods with inexact gradient information for finding stationary points of nonconvex continuously differentiable functions on finite-dimensional spaces. Some abstract convergence results for a…
Many recent applications in machine learning and data fitting call for the algorithmic solution of structured smooth convex optimization problems. Although the gradient descent method is a natural choice for this task, it requires exact…
A method of numerically evaluating slowly convergent monotone series is described. First, we apply a condensation transformation due to Van Wijngaarden to the original series. This transforms the original monotone series into an alternating…
Randomized Kaczmarz (RK), Motzkin Method (MM) and Sampling Kaczmarz Motzkin (SKM) algorithms are commonly used iterative techniques for solving a system of linear inequalities (i.e., $Ax \leq b$). As linear systems of equations represent a…
We give a complete characterization of the behavior of the Anderson acceleration (with arbitrary nonzero mixing parameters) on linear problems. Let n be the grade of the residual at the starting point with respect to the matrix defining the…
In the last decade, tensors have shown their potential as valuable tools for various tasks in numerical linear algebra. While most of the research has been focusing on how to compress a given tensor in order to maintain information as well…
Alternating minimization (AM) procedures are practically efficient in many applications for solving convex and non-convex optimization problems. On the other hand, Nesterov's accelerated gradient is theoretically optimal first-order method…
The objective of this paper is to understand the superlinear convergence behavior of the GMRES method when the coefficient matrix has clustered eigenvalues. In order to understand the phenomenon, we analyze the convergence using the…
Hardware trends have motivated the development of mixed precision algo-rithms in numerical linear algebra, which aim to decrease runtime while maintaining acceptable accuracy. One recent development is the development of an adaptive…
Preconditioned Krylov subspace (KSP) methods are widely used for solving large-scale sparse linear systems arising from numerical solutions of partial differential equations (PDEs). These linear systems are often nonsymmetric due to the…
We consider linear parameter-dependent systems $A(\mu) x(\mu) = b$ for many different $\mu$, where $A$ is large and sparse, and depends nonlinearly on $\mu$. Solving such systems individually for each $\mu$ would require great computational…
The problem of minimizing a separable convex function under linearly coupled constraints arises from various application domains such as economic systems, distributed control, and network flow. The main challenge for solving this problem is…
Motivated by an inertial primal-dual dynamical system with vanishing damping, we propose a class of accelerated augmented Lagrangian methods with Nesterov extrapolation parameters for a linearly constrained convex optimization problem with…
In this paper, we present a convergence rate analysis for the inexact Krasnosel'skii-Mann iteration built from nonexpansive operators. Our results include two main parts: we first establish global pointwise and ergodic iteration-complexity…
We analyze a modified version of Nesterov accelerated gradient algorithm, which applies to affine fixed point problems with non self-adjoint matrices, such as the ones appearing in the theory of Markov decision processes with discounted or…
Over the fast few years, the numerical success of the generalized alternating direction method of multipliers (GADMM) proposed by Eckstein \& Bertsekas [Math. Prog., 1992] has inspired intensive attention in analyzing its theoretical…
The nonlinear conjugate gradient methods are known to be an effective approach for standard unconstrained optimization problems especially for large-scale problems. This paper proposes a proximal nonlinear conjugate gradient method, which…