Related papers: Universality of Newton's method

We analyze the convergence of quasi-Newton methods in exact and finite precision arithmetic. In particular, we derive an upper bound for the stagnation level and we show that any sufficiently exact quasi-Newton method will converge…

We are interested in the ``smoothest'' averaging that can be achieved by convolving functions $f \in \ell^2(\mathbb{Z})$ with an averaging function $u$. More precisely, suppose $u:\{-n, \ldots, n\} \to \mathbb{R}$ is a symmetric function…

We will consider the damped Newton method for strongly monotone and Lipschitz continuous operator equations in a variational setting. We will provide a very accessible justification why the undamped Newton method performs better than its…

The nonlinear eigen-problem $ Ax+F(x)=\lambda x$ is studied where $A$ is an $n\times n$ irreducible Stieltjes matrix. Under certain conditions, this problem has a unique positive solution. We show that, starting from a multiple of the…

A class of algorithms comprised by certain semismooth Newton and active-set methods is able to solve convex minimization problems involving sparsity-inducing regularizers very rapidly; the speed advantage of methods from this class is a…

We study the composite convex optimization problems with a Quasi-Self-Concordant smooth component. This problem class naturally interpolates between classic Self-Concordant functions and functions with Lipschitz continuous Hessian.…

Fourier-Motzkin elimination, a standard method for solving systems of linear inequalities, leads to an elementary, short, and self-contained proof of von Neumann's minimax theorem.

We analyse the convergence of the gradient projection algorithm, which is finalized with the Newton method, to a stationary point for the problem of nonconvex constrained optimization $\min_{x \in S} f(x)$ with a proximally smooth set $S =…

While there already exist randomized subspace Newton methods that restrict the search direction to a random subspace for a convex function, we propose a randomized subspace regularized Newton method for a non-convex function {and more…

The method of the fundamental solutions (MFS) is used to construct an approximate solution for a partial differential equation in a bounded domain. It is demonstrated by combining the fundamental solutions shifted to the points outside the…

In this paper we derive a Newton type method to solve the non-linear system formed by combining the Tikhonov normal equations and Morozov's discrepancy principle. We prove that by placing a bound on the step size of the Newton iterations…

This paper is concerned with an algorithm for finding a singularity of the nonsmooth vector fields. Firstly, we discuss the main results of the Newton method presented in [1] for solving the aforementioned problem. Combining this method…

We consider the computation of stable approximations to the exact solution $x^\dag$ of nonlinear ill-posed inverse problems $F(x)=y$ with nonlinear operators $F:X\to Y$ between two Hilbert spaces $X$ and $Y$ by the Newton type methods $$…

The popular BFGS quasi-Newton minimization algorithm under reasonable conditions converges globally on smooth convex functions. This result was proved by Powell in 1976: we consider its implications for functions that are not smooth. In…

It has been widely recognized that the 0/1 loss function is one of the most natural choices for modelling classification errors, and it has a wide range of applications including support vector machines and 1-bit compressed sensing. Due to…

Let $F(u)=h$ be an operator equation in a Banach space $X$, $\|F'(u)-F'(v)\|\leq \omega(\|u-v\|)$, where $\omega\in C([0,\infty))$, $\omega(0)=0$, $\omega(r)>0$ if $r>0$, $\omega(r)$ is strictly growing on $[0,\infty)$. Denote…

The paper starts with a concise description of the recently developed semismooth* Newton method for the solution of general inclusions. This method is then applied to a class of variational inequalities of the second kind. As a result, one…

In the present paper, in order to fnd a singularity of a vector field defined on Riemannian manifolds, we present a new globalization strategy of Newton method and establish its global convergence with superlinear rate. In particular, this…

In this paper a special semi-smooth equation associated to the second order cone is studied. It is shown that, under mild assumptions, the semi-smooth Newton method applied to this equation is well-defined and the generated sequence is…

By using Moreau's decomposition theorem for projecting onto cones, the problem of projecting onto a simplicial cone is reduced to finding the unique solution of a nonsmooth system of equations. It is shown that a semi-smooth Newton method…