Related papers: Newton Nonholonomic Source Seeking for Distance-De…
A new decomposition optimization algorithm, called \textit{path-following gradient-based decomposition}, is proposed to solve separable convex optimization problems. Unlike path-following Newton methods considered in the literature, this…
Extremum seeking control (ESC) often employs perturbation-based estimates of derivatives for some sensor field or cost function. These estimates are generally obtained by simply multiplying the output of a single-unit sensor by some…
We propose a distributed cubic regularization of the Newton method for solving (constrained) empirical risk minimization problems over a network of agents, modeled as undirected graph. The algorithm employs an inexact, preconditioned Newton…
The Cartesian reverse derivative is a categorical generalization of reverse-mode automatic differentiation. We use this operator to generalize several optimization algorithms, including a straightforward generalization of gradient descent…
Many data-fitting applications require the solution of an optimization problem involving a sum of large number of functions of high dimensional parameter. Here, we consider the problem of minimizing a sum of $n$ functions over a convex…
We propose a new method that uses deep learning techniques to solve the inverse problems. The inverse problem is cast in the form of learning an end-to-end mapping from observed data to the ground-truth. Inspired by the splitting strategy…
In this paper new descent line search iterative schemes for unconstrained as well as constrained optimization problems are developed using q-derivative. At every iteration of the scheme, a positive definite matrix is provided which is…
We give a new improvement over Newton's method for root-finding, when the function in question is doubly differentiable. It generally exhibits faster and more reliable convergence. It can be also be thought of as a correction to Halley's…
Developing a contemporary optimal transport (OT) solver requires navigating trade-offs among several critical requirements: GPU parallelization, scalability to high-dimensional problems, theoretical convergence guarantees, empirical…
Algorithms based on the hard thresholding principle have been well studied with sounding theoretical guarantees in the compressed sensing and more general sparsity-constrained optimization. It is widely observed in existing empirical…
We consider an extension of the Newton-MR algorithm for nonconvex unconstrained optimization to the settings where Hessian information is approximated. Under a particular noise model on the Hessian matrix, we investigate the iteration and…
Blind source separation, i.e. extraction of independent sources from a mixture, is an important problem for both artificial and natural signal processing. Here, we address a special case of this problem when sources (but not the mixing…
The reciprocal square root is an important computation for which many sophisticated algorithms exist (see for example \cite{Moroz,863046,863031} and the references therein). A common theme is the use of Newton's method to refine the…
We consider point sources in hyperbolic equations discretized by finite differences. If the source is stationary, appropriate source discretization has been shown to preserve the accuracy of the finite difference method. Moving point…
We consider the solution of variational equations on manifolds by Newton's method. These problems can be expressed as root finding problems for mappings from infinite dimensional manifolds into dual vector bundles. We derive the…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…
In this paper, we study Newton-conjugate gradient (Newton-CG) methods for minimizing a nonconvex function $f$ whose Hessian is $(H_f,\nu)$-H\"older continuous with modulus $H_f>0$ and exponent $\nu\in(0,1]$. Recently proposed Newton-CG…
First order shape optimization methods, in general, require a large number of iterations until they reach a locally optimal design. While higher order methods can significantly reduce the number of iterations, they exhibit only local…
Some fractional Newton methods have been proposed in order to find roots of nonlinear equations using fractional derivatives. In this paper we introduce a fractional Newton method with order $\alpha+1$ and compare with another fractional…
Second-order methods are emerging as promising alternatives to standard first-order optimizers such as gradient descent and ADAM for training neural networks. Though the advantages of including curvature information in computing…