Related papers: Anderson Acceleration For Perturbed Newton Methods
Newton's method for polynomial root finding is one of mathematics' most well-known algorithms. The method also has its shortcomings: it is undefined at critical points, it could exhibit chaotic behavior and is only guaranteed to converge…
We present selected results from combined analytical and numerical studies of the Anderson-impurity model within the framework of infinite order perturbation theory with respect to the hybridization. Our approximation goes considerably…
When studying the multilinear PageRank problem, a system of polynomial equations needs to be solved. In this paper, we develop convergence theory for a modified Newton method in a particular parameter regime. The sequence of vectors…
Stochastic gradient descent (SGD) and its many variants are the widespread optimization algorithms for training deep neural networks. However, SGD suffers from inevitable drawbacks, including vanishing gradients, lack of theoretical…
In this paper, we first propose a new Levenberg-Marquardt method for solving constrained (and not necessarily square) nonlinear systems. Basically, the method combines the unconstrained Levenberg-Marquardt method with a type of feasible…
Dual descent methods are commonly used to solve network optimization problems because their implementation can be distributed through the network. However, their convergence rates are typically very slow. This paper introduces a family of…
Mathematical models for flow and reactive transport in porous media often involve non-linear, degenerate parabolic equations. Their solutions have low regularity, and therefore lower order schemes are used for the numerical approximation.…
We study the behaviour of solutions to a class of nonlinear degenerate parabolic problems when the data are perturbed. The class includes the Richards equation, Stefan problem and the parabolic $p$-Laplace equation. We show that, up to a…
We develop two new variants of alternating direction methods of multipliers (ADMM) and two parallel primal-dual decomposition algorithms to solve a wide range class of constrained convex optimization problems. Our approach relies on a novel…
We consider solving equality-constrained nonlinear, nonconvex optimization problems. This class of problems appears widely in a variety of applications in machine learning and engineering, ranging from constrained deep neural networks, to…
Many problems in geometric optics or convex geometry can be recast as optimal transport problems: this includes the far-field reflector problem, Alexandrov's curvature prescription problem, etc. A popular way to solve these problems…
We propose and analyze an Accelerated Rearrangement Method (ARM) for solving a class of nonconvex optimization problems involving two-phase composites. These problems include maximizing the (work) energy of a membrane governed by the…
We analyze the performance of a variant of Newton method with quadratic regularization for solving composite convex minimization problems. At each step of our method, we choose regularization parameter proportional to a certain power of the…
A pervasive approach in scientific computing is to express the solution to a given problem as the limit of a sequence of vectors or other mathematical objects. In many situations these sequences are generated by slowly converging iterative…
We give new convergence results of Anderson acceleration for the composite $\max$ fixed point problem. We prove that Anderson(1) and EDIIS(1) are q-linear convergent with a smaller q-factor than existing q-factors. Moreover, we propose a…
In this paper, we develop a novel primal-dual semismooth Newton method for solving linearly constrained multi-block convex composite optimization problems. First, a differentiable augmented Lagrangian (AL) function is constructed by…
As second-order methods, Gauss--Newton-type methods can be more effective than first-order methods for the solution of nonsmooth optimization problems with expensive-to-evaluate smooth components. Such methods, however, often do not…
We describe the large-time moment asymptotics for the parabolic Anderson model where the speed of the diffusion is coupled with time, inducing an acceleration or deceleration. We find a lower critical scale, below which the mass flow gets…
Nonlinear acceleration algorithms improve the performance of iterative methods, such as gradient descent, using the information contained in past iterates. However, their efficiency is still not entirely understood even in the quadratic…
In this paper a robust second-order method is developed for the solution of strongly convex l1-regularized problems. The main aim is to make the proposed method as inexpensive as possible, while even difficult problems can be efficiently…