Related papers: Newton-Anderson at Singular Points
Anderson acceleration (AA) is widely used for accelerating the convergence of nonlinear fixed-point methods $x_{k+1}=q(x_{k})$, $x_k \in \mathbb{R}^n$, but little is known about how to quantify the convergence acceleration provided by AA.…
In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression. We first prove that our new…
Inverse problems are in many cases solved with optimization techniques. When the underlying model is linear, first-order gradient methods are usually sufficient. With nonlinear models, due to nonconvexity, one must often resort to…
This paper investigates the use of fixed-point Anderson acceleration method (AA) to a recently proposed hierarchical control framework. Due to its model-free property, the AA-based resulting hierarchical framework becomes more generic since…
A technique for accelerating global convergence of pseudo-transient continuation Newton methods is proposed based on residual smoothing. The technique is motivated by the effectiveness of local nonlinear smoothers at overcoming strong…
Anderson mixing (AM) is an acceleration method for fixed-point iterations. Despite its success and wide usage in scientific computing, the convergence theory of AM remains unclear, and its applications to machine learning problems are not…
{A defining characteristic of Newton's method is local superlinear convergence within a neighbourhood of a strict local minimum. However, outside this neighborhood Newton's method can converge slowly or even diverge. A common approach to…
An implementation and an application of the combination of the genetic algorithm and Newton's method for solving a system of nonlinear equations is presented. The method first uses the advantage of the robustness of the genetic algorithm…
In this work, we propose a nonlinear stabilization technique for scalar conservation laws with implicit time stepping. The method relies on an artificial diffusion method, based on a graph-Laplacian operator. It is nonlinear, since it…
Minimizing loss functions is central to machine-learning training. Although first-order methods dominate practical applications, higher-order techniques such as Newton's method can deliver greater accuracy and faster convergence, yet are…
In this paper, we propose new methods to efficiently solve convex optimization problems encountered in sparse estimation, which include a new quasi-Newton method that avoids computing the Hessian matrix and improves efficiency, and we prove…
This paper proposes and develops new Newton-type methods to solve structured nonconvex and nonsmooth optimization problems with justifying their fast local and global convergence by means of advanced tools of variational analysis and…
The nonlinear systems obtained by discretizing degenerate parabolic equations may be hard to solve, especially with Newton's method. In this paper, we apply to Richards equation a strategy that consists in defining a new primary unknown for…
In this work, we extend a modified Anderson acceleration proposed in [Y. He, arXiv:2603.25983, 2026] to accelerate the Picard iteration for the Navier-Stokes equations. In this variant of Anderson acceleration, named AAg, the nonlinear…
We consider the problem of finding the best approximation point from a polyhedral set, and its applications, in particular to solving large-scale linear programs. The classical projection problem has many various and many applications. We…
In this paper we develop an adaptive procedure for the numerical solution of semilinear parabolic problems, with possible singular perturbations. Our approach combines a linearization technique using Newton's method with an adaptive…
We exploit analogies between first-order algorithms for constrained optimization and non-smooth dynamical systems to design a new class of accelerated first-order algorithms for constrained optimization. Unlike Frank-Wolfe or projected…
A new Jacobian approximation is developed for use in quasi-Newton methods for solving systems of nonlinear equations. The new hypersecant Jacobian approximation is intended for the special case where the evaluation of the functions whose…
In this work we study the problem of unconstrained convex-optimization in a fully distributed multi-agent setting which includes asynchronous computation and lossy communication. In particular, we extend a recently proposed algorithm named…
In this paper, we propose a quasi-Newton method for solving smooth and monotone nonlinear equations, including unconstrained minimization and minimax optimization as special cases. For the strongly monotone setting, we establish two global…