Related papers: Anderson Acceleration Based on the $\mathcal{H}^{-…
Optimization plays a key role in machine learning. Recently, stochastic second-order methods have attracted much attention due to their low computational cost in each iteration. However, these algorithms might perform poorly especially if…
Recently, the anchor acceleration, an acceleration mechanism distinct from Nesterov's, has been discovered for minimax optimization and fixed-point problems, but its mechanism is not understood well, much less so than Nesterov acceleration.…
Electrical Impedance Tomography (EIT) is widely applied in medical diagnosis, industrial inspection, and environmental monitoring. Combining the physical principles of the imaging system with the advantages of data-driven deep learning…
In this paper, the Newton-Anderson method, which results from applying an extrapolation technique known as Anderson acceleration to Newton's method, is shown both analytically and numerically to provide superlinear convergence to non-simple…
We propose accelerated versions of the operator Sinkhorn iteration for operator scaling using successive overrelaxation. We analyze the local convergence rates of these accelerated methods via linearization, which allows us to determine the…
In this paper we consider the neural network optimization. We develop Anderson-type acceleration method for the stochastic gradient decent method and it improves the network permanence very much. We demonstrate the applicability of the…
BoostConv has been introduced in earlier works as an effective acceleration technique for nonlinear iterative processes and has been successfully employed in a variety of applications to enhance convergence rates or to compute unstable…
Static analysis by abstract interpretation aims at automatically proving properties of computer programs. To do this, an over-approximation of program semantics, defined as the least fixpoint of a system of semantic equations, must be…
Acceleration of first order methods is mainly obtained via inertial techniques \`a la Nesterov, or via nonlinear extrapolation. The latter has known a recent surge of interest, with successful applications to gradient and proximal gradient…
In this paper, we study the robust linearization of nonlinear poromechanics of unsaturated materials. The model of interest couples the Richards equation with linear elasticity equations, employing the equivalent pore pressure. In practice…
We give a complete characterization of the behavior of the Anderson acceleration (with arbitrary nonzero mixing parameters) on linear problems. Let n be the grade of the residual at the starting point with respect to the matrix defining the…
Predicting the behavior of a magnetically confined fusion plasma over long time periods requires methods that can bridge the difference between transport and turbulent time scales. The nonlinear transport solver, Tango, enables simulations…
The Anderson Mixing (AM) method is a popular approach for accelerating fixed-point iterations by leveraging historical information from previous steps. In this paper, we introduce the Riemannian Anderson Mixing (RAM) method, an extension of…
Despite their frequent slow convergence, proximal gradient schemes are widely used in large-scale optimization tasks due to their tremendous stability, scalability, and ease of computation. In this paper, we develop and investigate a…
As first-order optimization methods become the method of choice for solving large-scale optimization problems, optimization solvers based on first-order algorithms are being built. Such general-purpose solvers must robustly detect…
The atomic norm provides a generalization of the $\ell_1$-norm to continuous parameter spaces. When applied as a sparse regularizer for line spectral estimation the solution can be obtained by solving a convex optimization problem. This…
Multilinear systems play an important role in scientific calculations of practical problems. In this paper, we consider a tensor splitting method with a relaxed Anderson acceleration for solving multilinear systems. The new method preserves…
We propose Acc-Sinkhorn, a simple accelerated variant of Sinkhorn for entropy-regularized optimal transport (EOT). The method is derived from a bilevel optimization view: Sinkhorn row scaling solves the inner variable $u$ exactly and…
In this paper, we develop a novel accelerated fixed-point-based framework using delayed inexact oracles to approximate a fixed point of a nonexpansive operator (or equivalently, a root of a co-coercive operator), a central problem in…
This work introduces a moving anchor acceleration technique to extragradient algorithms for smooth structured minimax problems. The moving anchor is introduced as a generalization of the original algorithmic anchoring framework, i.e. the…