相关论文: Nonmonotone Globalization for Anderson Acceleratio…

In [19], a general, inexact, efficient proximal quasi-Newton algorithm for composite optimization problems has been proposed and a sublinear global convergence rate has been established. In this paper, we analyze the convergence properties…

We analyze the performance of a variant of Newton method with quadratic regularization for solving composite convex minimization problems. At each step of our method, we choose regularization parameter proportional to a certain power of the…

We consider the Adaptive Antoulas-Anderson (AAA) rational interpolation algorithm recently developed by Trefethen and co-authors, which can be viewed as a type of moment-matching technique for system realization and approximation. We…

This paper describes the design of a safeguarding scheme for Anderson acceleration to improve its practical performance and stability when used for first-order optimisation methods. We show how the combination of a non-expansiveness…

This paper primarily presents numerical results for the Anderson accelerated Newton method on a set of benchmark problems. The results demonstrate superlinear convergence to solutions of both degenerate and nondegenerate problems. The…

The DANE algorithm is an approximate Newton method popularly used for communication-efficient distributed machine learning. Reasons for the interest in DANE include scalability and versatility. Convergence of DANE, however, can be tricky;…

This paper presents the design and development of an Anderson Accelerated Preconditioned Modified Hermitian and Skew-Hermitian Splitting (AA-PMHSS) method for solving complex-symmetric linear systems with application to electromagnetics…

Non-monotone trust-region methods are known to provide additional benefits for scalar and multi-objective optimization, such as enhancing the probability of convergence and improving the speed of convergence. For optimization of set-valued…

In this paper, we propose an acceleration framework for a class of iterative methods using the Reduced Order Method (ROM). Assuming that the underlying iterative scheme generates a rich basis for the solution space, we construct the next…

In this paper, we introduce the Adaptive Inertial Method (AIM), a novel framework for accelerated first-order methods through a customizable inertial term. We provide a rigorous convergence analysis establishing a global convergence rate of…

The trust-region (TR) method is renowned historically for its robustness in nonconvex problems and extraordinary numerical performance, but the study of its performance in convex optimization is somehow limited. This paper complements the…

This paper considers an enhancement of the classical iterated penalty Picard (IPP) method for the incompressible Navier-Stokes equations, where we restrict our attention to $O(1)$ penalty parameter, and Anderson acceleration (AA) is used to…

The adaptive Antoulas-Anderson (AAA) algorithm for rational approximation is a widely used method for the efficient construction of highly accurate rational approximations to given data. While AAA can often produce rational approximations…

While the most recent Anderson acceleration (AA) convergence theory [Pollock et al, {\it IMA Num. An.}, 2021] requires that the AA optimization norm match the Hilbert space norm associated with the fixed point operator, in implementations…

Decentralized optimization is a powerful paradigm that finds applications in engineering and learning design. This work studies decentralized composite optimization problems with non-smooth regularization terms. Most existing gradient-based…

We study stochastic second-order methods for solving general non-convex optimization problems. We propose using a special version of momentum to stabilize the stochastic gradient and Hessian estimates in Newton's method. We show that…

The Carleman embedding method is a widely used technique for linearizing a system of nonlinear differential equations, but fails to converge in regions where there are multiple fixed points. We propose and test three different versions of a…

In this paper we explore acceleration techniques for large scale nonconvex optimization problems with special focuses on deep neural networks. The extrapolation scheme is a classical approach for accelerating stochastic gradient descent for…

We investigate the proximal point algorithm (PPA) and its inexact extensions under an error bound condition, which guarantees a global linear convergence if the proximal regularization parameter is larger than the error bound condition…

This paper presents a novel accelerated distributed algorithm for unconstrained consensus optimization over static undirected networks. The proposed algorithm combines the benefits of acceleration from momentum, the robustness of the…