Related papers: An improved convergence theorem for the Newton met…
One of the most important problems in the field of distributed optimization is the problem of minimizing a sum of local convex objective functions over a networked system. Most of the existing work in this area focus on developing…
The motive of this paper is to discuss the local convergence of a two-step Newton type method of convergence rate three for solving nonlinear equations in Banach spaces. It is assumed that the first order derivative of nonlinear operator…
A regularization algorithm (AR1pGN) for unconstrained nonlinear minimization is considered, which uses a model consisting of a Taylor expansion of arbitrary degree and regularization term involving a possibly non-smooth norm. It is shown…
Motivated by robotic trajectory optimization problems we consider the Augmented Lagrangian approach to constrained optimization. We first propose an alternative augmentation of the Lagrangian to handle the inequality case (not based on…
In the first part of this paper, we establish a conditional optimality result for an adaptive mixed finite element method for the stationary Stokes problem discretized by the standard Taylor-Hood elements, under the assumption of the…
We propose two variants of Newton method for solving unconstrained minimization problem. Our method leverages optimization techniques such as penalty and augmented Lagrangian method to generate novel variants of the Newton method namely the…
This paper presents a regularized Newton method (RNM) with generalized regularization terms for unconstrained convex optimization problems. The generalized regularization includes quadratic, cubic, and elastic net regularizations as special…
Majorization-minimization schemes are a broad class of iterative methods targeting general optimization problems, including nonconvex, nonsmooth and stochastic. These algorithms minimize successively a sequence of upper bounds of the…
We study composite optimization problems in which the smooth part of the objective function is \( p \)-times continuously differentiable, where \( p \geq 1 \) is an integer. Higher-order methods are known to be effective for solving such…
We study the application of the Augmented Lagrangian Method to the solution of linear ill-posed problems. Previously, linear convergence rates with respect to the Bregman distance have been derived under the classical assumption of a…
Clustering may be the most fundamental problem in unsupervised learning which is still active in machine learning research because its importance in many applications. Popular methods like K-means, may suffer from instability as they are…
It has been widely recognized that the 0/1 loss function is one of the most natural choices for modelling classification errors, and it has a wide range of applications including support vector machines and 1-bit compressed sensing. Due to…
We propose a second-order method for unconditional minimization of functions $f(z)$ of complex arguments. We call it the Mixed Newton Method due to the use of the mixed Wirtinger derivative $\frac{\partial^2f}{\partial\bar z\partial z}$ for…
We show that a second order sufficient condition for local optimality, along with a strict complementarity condition, is enough to get the superlinear convergence of the semismooth Newton method for an optimal control problem governed by a…
We consider the efficient minimization of a nonlinear, strictly convex functional with $\ell_1$-penalty term. Such minimization problems appear in a wide range of applications like Tikhonov regularization of (non)linear inverse problems…
The goal of this paper is to study approaches to bridge the gap between first-order and second-order type methods for composite convex programs. Our key observations are: i) Many well-known operator splitting methods, such as…
At the heart of Newton based optimization methods is a sequence of symmetric linear systems. Each consecutive system in this sequence is similar to the next, so solving them separately is a waste of computational effort. Here we describe…
While there already exist randomized subspace Newton methods that restrict the search direction to a random subspace for a convex function, we propose a randomized subspace regularized Newton method for a non-convex function {and more…
For first-order optimization of non-convex functions with Lipschitz continuous gradient and Hessian, the best known complexity for reaching an $\varepsilon$-approximation of a stationary point is $\mathcal{O}(\varepsilon^{-7/4})$. Existing…
We consider (stochastic) subgradient methods for strongly convex but potentially nonsmooth non-Lipschitz optimization. We provide new equivalent dual descriptions (in the style of dual averaging) for the classic subgradient method, the…