Related papers: Action constrained quasi-Newton methods
We propose a Forward-Backward Truncated-Newton method (FBTN) for minimizing the sum of two convex functions, one of which smooth. Unlike other proximal Newton methods, our approach does not involve the employment of variable metrics, but is…
Diagonal preconditioners are computationally feasible approximate to second-order optimizers, which have shown significant promise in accelerating training of deep learning models. Two predominant approaches are based on Adam and…
The impact of different linearisation and iterative solution strategies for fully-coupled pressure-based algorithms for compressible flows at all speeds is studied, with the aim of elucidating their impact on the performance of the…
A technique for accelerating global convergence of pseudo-transient continuation Newton methods is proposed based on residual smoothing. The technique is motivated by the effectiveness of local nonlinear smoothers at overcoming strong…
This paper considers the regularization continuation method and the trust-region updating strategy for the nonlinearly equality-constrained optimization problem. Namely, it uses the inverse of the regularization quasi-Newton matrix as the…
We analyze the conjugate gradient (CG) method with variable preconditioning for solving a linear system with a real symmetric positive definite (SPD) matrix of coefficients $A$. We assume that the preconditioner is SPD on each step, and…
We study a class of monotone inclusions called "self-concordant inclusion" which covers three fundamental convex optimization formulations as special cases. We develop a new generalized Newton-type framework to solve this inclusion. Our…
As second-order methods, Gauss--Newton-type methods can be more effective than first-order methods for the solution of nonsmooth optimization problems with expensive-to-evaluate smooth components. Such methods, however, often do not…
We propose a novel preconditioned inexact primal-dual interior point method for constrained convex quadratic programming problems. The algorithm we describe invokes the preconditioned conjugate gradient method on a new reduced Schur…
We propose an alternative implementation of preconditioning techniques for the solution of non-linear problems. Within the framework of Newton-Krylov methods, preconditioning techniques are needed to improve the performance of the solvers.…
We study first-order methods with preconditioning for solving structured nonlinear convex optimization problems. We propose a new family of preconditioners generated by symmetric polynomials. They provide first-order optimization methods…
This paper is concerned with the convergence of a two-step modified Newton method for solving the nonlinear system arising from the minimal nonnegative solution of nonsymmetric algebraic Riccati equations from neutron transport theory. We…
The iterative problem of solving nonlinear equations is studied. A new Newton like iterative method with adjustable parameters is designed based on the dynamic system theory. In order to avoid the derivative function in the iterative…
We consider distributed optimization problems where networked nodes cooperatively minimize the sum of their locally known convex costs. A popular class of methods to solve these problems are the distributed gradient methods, which are…
This paper concerns exact linesearch quasi-Newton methods for minimizing a quadratic function whose Hessian is positive definite. We show that by interpreting the method of conjugate gradients as a particular exact linesearch quasi-Newton…
We propose a randomized second-order method for optimization known as the Newton Sketch: it is based on performing an approximate Newton step using a randomly projected or sub-sampled Hessian. For self-concordant functions, we prove that…
We study a variant of Newton's algorithm applied to under-determined systems of non-smooth equations. The notion of regularity employed in our work is based on Newton differentiability, which generalizes semi-smoothness. The classic notion…
The aim of this paper is to design computationally-efficient and optimal algorithms for the online and stochastic exp-concave optimization settings. Typical algorithms for these settings, such as the Online Newton Step (ONS), can guarantee…
The recently introduced Gradient Methods with Memory use a subset of the past oracle information to create an accurate model of the objective function that enables them to surpass the Gradient Method in practical performance. The model…
Quasi-Newton methods refer to a class of algorithms at the interface between first and second order methods. They aim to progress as substantially as second order methods per iteration, while maintaining the computational complexity of…