Related papers: How accurate does Newton have to be?
Considered herein is a modified Newton method for the numerical solution of nonlinear equations where the Jacobian is approximated using a complex-step derivative approximation. We show that this method converges for sufficiently small…
We study the local convergence of classical quasi-Newton methods for nonlinear optimization. Although it was well established a long time ago that asymptotically these methods converge superlinearly, the corresponding rates of convergence…
Quasi-Newton algorithms are among the most popular iterative methods for solving unconstrained minimization problems, largely due to their favorable superlinear convergence property. However, existing results for these algorithms are…
We introduce the primal-dual quasi-Newton (PD-QN) method as an approximated second order method for solving decentralized optimization problems. The PD-QN method performs quasi-Newton updates on both the primal and dual variables of the…
Newton-type methods enjoy fast local convergence and strong empirical performance, but achieving global guarantees comparable to first-order methods remains challenging. Even for simple strongly convex problems, no straightforward variant…
We give computational results to study the accuracy of several quasicontinuum methods for two benchmark problems - the stability of a Lomer dislocation pair under shear and the stability of a lattice to plastic slip under tensile loading.…
We introduce a framework for quasi-Newton forward--backward splitting algorithms (proximal quasi-Newton methods) with a metric induced by diagonal $\pm$ rank-$r$ symmetric positive definite matrices. This special type of metric allows for a…
The aim of this paper is to present a streamlined and fully three-dimensional version of the quasicontinuum (QC) theory of Tadmor et al. and to analyze its accuracy and convergence characteristics. Specifically, we assess the effect of the…
Optimization is important in machine learning problems, and quasi-Newton methods have a reputation as the most efficient numerical schemes for smooth unconstrained optimization. In this paper, we consider the explicit superlinear…
Linear programs with quadratic regularization are attracting renewed interest due to their applications in optimal transport: unlike entropic regularization, the squared-norm penalty gives rise to sparse approximations of optimal transport…
A general class of Newton algorithms on Gra{\ss}mann and Lagrange-Gra{\ss}mann manifolds is introduced, that depends on an arbitrary pair of local coordinates. Local quadratic convergence of the algorithm is shown under a suitable condition…
Optimal balance is a non-asymptotic numerical method to compute a point on the slow manifold for certain two-scale dynamical systems. It works by solving a modified version of the system as a boundary value problem in time, where the…
This paper pursues a twofold goal. First, we introduce and study in detail a new notion of variational analysis called generalized metric subregularity, which is a far-going extension of the conventional metric subregularity conditions. Our…
Newton's method is the most widespread high-order method, demanding the gradient and the Hessian of the objective function. However, one of the main disadvantages of Newtons method is its lack of global convergence and high iteration cost.…
Classification problems using deep learning have been shown to have a high-curvature subspace in the loss landscape equal in dimension to the number of classes. Moreover, this subspace corresponds to the subspace spanned by the logit…
Stability issues with reinforcement learning methods persist. To better understand some of these stability and convergence issues involving deep reinforcement learning methods, we examine a simple linear quadratic example. We interpret the…
For solving large-scale non-convex problems, we propose inexact variants of trust region and adaptive cubic regularization methods, which, to increase efficiency, incorporate various approximations. In particular, in addition to approximate…
In this paper the simplicial cone constrained convex quadratic programming problem is studied. The optimality conditions of this problem consist in a linear complementarity problem. This fact, under a suitable condition, leads to an…
Several algorithms in computer algebra involve the computation of a power series solution of a given ordinary differential equation. Over finite fields, the problem is often lifted in an approximate $p$-adic setting to be well-posed. This…
Many machine learning models depend on solving a large scale optimization problem. Recently, sub-sampled Newton methods have emerged to attract much attention for optimization due to their efficiency at each iteration, rectified a weakness…