Related papers: Anderson Acceleration For Perturbed Newton Methods
A one-step analysis of Anderson acceleration with general algorithmic depths is presented. The resulting residual bounds within both contractive and noncontractive settings reveal the balance between the contributions from the higher and…
Anderson localization is a regime in which diffusion is inhibited and waves (also electromagnetic waves) get localized. Here we exploit adaptive optics to achieve focusing in disordered optical fibers in the Anderson regime. By wavefront…
We propose two variants of Newton method for solving unconstrained minimization problem. Our method leverages optimization techniques such as penalty and augmented Lagrangian method to generate novel variants of the Newton method namely the…
This paper proposes a new framework for computing low-rank solutions to nonlinear matrix equations arising from spatial discretization of nonlinear partial differential equations: low-rank Anderson acceleration (lrAA). lrAA is an adaptation…
We propose LeAP-SSN (Levenberg--Marquardt Adaptive Proximal Semismooth Newton method), a semismooth Newton-type method with a simple, parameter-free globalisation strategy that guarantees convergence from arbitrary starting points in…
This paper develops a new class of nonlinear acceleration algorithms based on extending conjugate residual-type procedures from linear to nonlinear equations. The main algorithm has strong similarities with Anderson acceleration as well as…
This paper considers the numerical solution of generalized Sylvester matrix equations, which arise in many scientific and engineering applications but remain challenging to solve efficiently, particularly when the coefficient matrices are…
The properties of current-carrying steady states of strongly correlated systems away from the linear-response regime are of topical interest. In this article, we review the renormalized perturbation theory, or renormalized SPT of reference…
In this paper, we consider the Anderson acceleration method for solving the contractive fixed point problem, which is nonsmooth in general. We define a class of smoothing functions for the original nonsmooth fixed point mapping, which can…
Iteratively reweighted L1 (IRL1) algorithm is a common algorithm for solving sparse optimization problems with nonconvex and nonsmooth regularization. The development of its acceleration algorithm, often employing Nesterov acceleration, has…
We propose NAMA (Newton-type Alternating Minimization Algorithm) for solving structured nonsmooth convex optimization problems where the sum of two functions is to be minimized, one being strongly convex and the other composed with a linear…
The self-consistent theory of Anderson localization of quantum particles or classical waves in disordered media is reviewed. After presenting the basic concepts of the theory of Anderson localization in the case of electrons in disordered…
Optimization plays a key role in machine learning. Recently, stochastic second-order methods have attracted much attention due to their low computational cost in each iteration. However, these algorithms might perform poorly especially if…
Historically speaking, it is hard to balance the global and local efficiency of second-order optimization algorithms. For instance, the classical Newton's method possesses excellent local convergence but lacks global guarantees, often…
Newton's method may exhibit slower convergence than vanilla Gradient Descent in its initial phase on strongly convex problems. Classical Newton-type multilevel methods mitigate this but, like Gradient Descent, achieve only linear…
Newton's method may exhibit slower convergence than vanilla Gradient Descent in its initial phase on strongly convex problems. Classical Newton-type multilevel methods mitigate this but, like Gradient Descent, achieve only linear…
Despite their frequent slow convergence, proximal gradient schemes are widely used in large-scale optimization tasks due to their tremendous stability, scalability, and ease of computation. In this paper, we develop and investigate a…
This work studies the linear convergence of an accelerated scheme of the Alternating Direction Method of Multipliers (ADMM) for strongly convex and Lipschitz-smooth problems. We use the methodology of expressing the accelerated ADMM as a…
The Anderson Mixing (AM) method is a popular approach for accelerating fixed-point iterations by leveraging historical information from previous steps. In this paper, we introduce the Riemannian Anderson Mixing (RAM) method, an extension of…
Nonconvex and structured optimization problems arise in many engineering applications that demand scalable and distributed solution methods. The study of the convergence properties of these methods is in general difficult due to the…