相关论文: Generalized continuation Newton methods and the tr…

This paper presents a regularized Newton method (RNM) with generalized regularization terms for unconstrained convex optimization problems. The generalized regularization includes quadratic, cubic, and elastic net regularizations as special…

The Newton iteration is a popular method for minimising a cost function on Euclidean space. Various generalisations to cost functions defined on manifolds appear in the literature. In each case, the convergence rate of the generalised…

We study and derive algorithms for nonlinear eigenvalue problems, where the system matrix depends on the eigenvector, or several eigenvectors (or their corresponding invariant subspace). The algorithms are derived from an implicit…

In this paper, a globally convergent Newton-type proximal gradient method is developed for composite multi-objective optimization problems where each objective function can be represented as the sum of a smooth function and a nonsmooth…

A new class of affine scaling matrices for the interior point Newton-type methods is considered to solve the nonlinear systems with simple bounds. We review the essential properties of a scaling matrix and consider several well-known…

In this article, we consider solvers for large-scale trust-region subproblems when the quadratic model is defined by a limited-memory symmetric rank-one (L-SR1) quasi-Newton matrix. We propose a solver that exploits the compact…

Solving complex optimization problems in engineering and the physical sciences requires repetitive computation of multi-dimensional function derivatives. Commonly, this requires computationally-demanding numerical differentiation such as…

In [19], a general, inexact, efficient proximal quasi-Newton algorithm for composite optimization problems has been proposed and a sublinear global convergence rate has been established. In this paper, we analyze the convergence properties…

Globally convergent variants of the Gauss-Newton algorithm are often the methods of choice to tackle nonlinear least-squares problems. Among such frameworks, Levenberg-Marquardt and trust-region methods are two well-established, similar…

This paper primarily presents numerical results for the Anderson accelerated Newton method on a set of benchmark problems. The results demonstrate superlinear convergence to solutions of both degenerate and nondegenerate problems. The…

In this paper, we present a Newton-like method based on model reduction techniques, which can be used in implicit numerical methods for approximating the solution to ordinary differential equations. In each iteration, the Newton-like method…

This paper investigates the global convergence of stepsized Newton methods for convex functions with H\"older continuous Hessians or third derivatives. We propose several simple stepsize schedules with fast global convergence guarantees, up…

Many machine learning models involve solving optimization problems. Thus, it is important to deal with a large-scale optimization problem in big data applications. Recently, subsampled Newton methods have emerged to attract much attention…

In this paper, we propose a distributed Newton method for consensus optimization. Our approach outperforms state-of-the-art methods, including ADMM. The key idea is to exploit the sparsity of the dual Hessian and recast the computation of…

By using the Hadamard matrix product concept, this paper introduces two generalized matrix formulation forms of numerical analogue of nonlinear differential operators. The SJT matrix-vector product approach is found to be a simple,…

Second-order methods are provably faster than first-order methods, and their efficient implementations for large-scale optimization problems have attracted significant attention. Yet, optimization problems in ML often have nonsmooth…

The Gauss-Newton matrix is widely viewed as a positive semidefinite approximation of the Hessian, yet mounting empirical evidence shows that Gauss-Newton descent outperforms Newton's method. We adopt a function space perspective to analyze…

We present a numerical method to efficiently solve optimization problems governed by large-scale nonlinear systems of equations, including discretized partial differential equations, using projection-based reduced-order models accelerated…

A local convergence analysis of the Gauss-Newton method for solving injective-overdetermined systems of nonlinear equations under a majorant condition is provided. The convergence as well as results on its rate are established without a…

In this paper we propose an efficiently preconditioned Newton method for the computation of the leftmost eigenpairs of large and sparse symmetric positive definite matrices. A sequence of preconditioners based on the BFGS update formula is…