Related papers: Adaptive Newton-Type Schemes Based on Projections
We proposed in this paper a new method, which we named the W4 method, to solve nonlinear equation systems. It may be regarded as an extension of the Newton-Raphson~(NR) method to be used when the method fails. Indeed our method can be…
We consider two formulations for distributed optimization wherein $N$ agents in a generic connected network solve a problem of common interest: distributed personalized optimization and consensus optimization. A new method termed DINAS…
We propose a trust-region method for finite-sum minimization with an adaptive sample size adjustment technique, which is practical in the sense that it leads to a globally convergent method that shows strong performance empirically without…
In this letter, we introduce a new generalized linearizing transformation (GLT) for second order nonlinear ordinary differential equations (SNODEs). The well known invertible point (IPT) and non-point transformations (NPT) can be derived as…
In this paper, we propose and analyze some practical Newton methods for electronic structure calculations. We show the convergence and the local quadratic convergence rate for the Newton method when the Newton search directions are…
The objective of this study is to address the difficulty of simplifying the geometric model in which a differential problem is formulated, also called defeaturing, while simultaneously ensuring that the accuracy of the solution is…
This paper presents a novel method of global adaptive dynamic programming (ADP) for the adaptive optimal control of nonlinear polynomial systems. The strategy consists of relaxing the problem of solving the Hamilton-Jacobi-Bellman (HJB)…
This paper considers the distributed optimization problem where each node of a peer-to-peer network minimizes a finite sum of objective functions by communicating with its neighboring nodes. In sharp contrast to the existing literature…
Many algorithms and applications involve repeatedly solving variations of the same inference problem; for example we may want to introduce new evidence to the model or perform updates to conditional dependencies. The goal of adaptive…
We investigate the use of regularized Newton methods with adaptive norms for optimizing neural networks. This approach can be seen as a second-order counterpart of adaptive gradient methods, which we here show to be interpretable as…
We develop a randomized Newton method capable of solving learning problems with huge dimensional feature spaces, which is a common setting in applications such as medical imaging, genomics and seismology. Our method leverages randomized…
Generalized alternating projections is an algorithm that alternates relaxed projections onto a finite number of sets to find a point in their intersection. We consider the special case of two linear subspaces, for which the algorithm…
Variable projection solves structured optimization problems by completely minimizing over a subset of the variables while iterating over the remaining variables. Over the last 30 years, the technique has been widely used, with empirical and…
The object of the present paper is to extend the third-order iterative method for solving nonlinear equations into systems of nonlinear equations. Since our motive is to develop the method which improve the order of convergence of Newton's…
In this paper, we propose a first second-order scheme based on arbitrary non-Euclidean norms, incorporated by Bregman distances. They are introduced directly in the Newton iterate with regularization parameter proportional to the square…
Recent studies have shown that many nonconvex machine learning problems satisfy a generalized-smooth condition that extends beyond traditional smooth nonconvex optimization. However, the existing algorithms are not fully adapted to such…
In this paper we the formulation of inverse problems as constrained minimization problems and their iterative solution by gradient or Newton type. We carry out a convergence analysis in the sense of regularization methods and discuss…
We consider empirical risk minimization for large-scale datasets. We introduce Ada Newton as an adaptive algorithm that uses Newton's method with adaptive sample sizes. The main idea of Ada Newton is to increase the size of the training set…
We consider a class of difference-of-convex (DC) optimization problems where the objective function is the sum of a smooth function and a possible nonsmooth DC function. The application of proximal DC algorithms to address this problem…
The parallel solution of large scale non-linear programming problems, which arise for example from the discretization of non-linear partial differential equations, is a highly demanding task. Here, a novel solution strategy is presented,…