Related papers: A more direct and better variant of New Q-Newton's…
This study introduces two second-order methods designed to provably avoid saddle points in composite nonconvex optimization problems: (i) a nonsmooth trust-region method and (ii) a curvilinear linesearch method. These developments are…
Quasi-Newton methods refer to a class of algorithms at the interface between first and second order methods. They aim to progress as substantially as second order methods per iteration, while maintaining the computational complexity of…
We focus here on a class of fourth-order parabolic equations that can be written as a system of second-order equations by introducing an auxiliary variable. We design a novel second-order fully discrete mixed finite element method to…
In this paper we take a quasi-Newton approach to nonlinear eigenvalue problems (NEPs) of the type $M(\lambda)v=0$, where $M:\mathbb{C}\rightarrow\mathbb{C}^{n\times n}$ is a holomorphic function. We investigate which types of approximations…
Non-negative matrix factorization (NMF) is a fundamental non-convex optimization problem with numerous applications in Machine Learning (music analysis, document clustering, speech-source separation etc). Despite having received extensive…
A new variant of Newton's method - named Backtracking New Q-Newton's method (BNQN) - which has strong theoretical guarantee, is easy to implement, and has good experimental performance, was recently introduced by the third author.…
This work is on constrained large-scale non-convex optimization where the constraint set implies a manifold structure. Solving such problems is important in a multitude of fundamental machine learning tasks. Recent advances on Riemannian…
In this paper, two numerical approaches based on the Newton iteration method with spectral algorithms are introduced to solve the Thomas-Fermi equation. That Thomas-Fermi equation is a nonlinear singular ordinary differential equation (ODE)…
In this paper, we are concerned with the numerical solution for the backward fractional Feynman-Kac equation with non-smooth initial data. Here we first provide the regularity estimate of the solution. And then we use the backward Euler and…
Quasi-Newton methods form an important class of methods for solving nonlinear optimization problems. In such methods, first order information is used to approximate the second derivative. The aim is to mimic the fast convergence that can be…
We give quantum speedups of several general-purpose numerical optimisation methods for minimising a function $f:\mathbb{R}^n \to \mathbb{R}$. First, we show that many techniques for global optimisation under a Lipschitz constraint can be…
In this paper, we introduce some new iterative optimisation algorithms on Riemannian manifolds and Hilbert spaces which have good global convergence guarantees to local minima. More precisely, these algorithms have the following properties:…
Machine learning problems such as neural network training, tensor decomposition, and matrix factorization, require local minimization of a nonconvex function. This local minimization is challenged by the presence of saddle points, of which…
A new Monte-Carlo method for solving linear parabolic partial differential equations is presented. Since, in this new scheme, the particles are followed backward in time, it provides great flexibility in choosing critical points in…
The forward-backward splitting method (FBS) for minimizing a nonsmooth composite function can be interpreted as a (variable-metric) gradient method over a continuously differentiable function which we call forward-backward envelope (FBE).…
A new continuous regularized Gauss-Newton-type method with simultaneous updates of the operator $(F^{\pr*}(x(t))F'(x(t))+\ep(t) I)^{-1}$ for solving nonlinear ill-posed equations in a Hilbert space is proposed. A convergence theorem is…
We present and investigate a new type of implicit fractional linear multistep method of order two for fractional initial value problems. The method is obtained from the second order super convergence of the Gr\"unwald-Letnikov approximation…
In this paper, we propose a variant of Riemannian stochastic recursive gradient method that can achieve second-order convergence guarantee and escape saddle points using simple perturbation. The idea is to perturb the iterates when gradient…
Retraction-free approaches offer attractive low-cost alternatives to Riemannian methods on the Stiefel manifold, but they are often first-order, which may limit the efficiency under high-accuracy requirements. To this end, we propose a…
In this article, we introduce two families of novel fractional $\theta$-methods by constructing some new generating functions to discretize the Riemann-Liouville fractional calculus operator $\mathit{I}^{\alpha}$ with a second order…