Related papers: Accelerating Newton-Schulz Iteration for Orthogona…
We propose a Riemannian limited-memory BFGS method for optimization problems with Euclidean bounds. The method combines a limited-memory quasi-Newton update in the tangent space with a Riemannian adaptation of the generalized Cauchy point…
Chebyshev interpolation is a highly effective, intensively studied method and enjoys excellent numerical properties. The interpolation nodes are known beforehand, implementation is straightforward and the method is numerically stable. For…
An algorithm for solving smooth nonconvex optimization problems is proposed that, in the worst-case, takes $\mathcal{O}(\epsilon^{-3/2})$ iterations to drive the norm of the gradient of the objective function below a prescribed positive…
A q-Gauss-Newton algorithm is an iterative procedure that solves nonlinear unconstrained optimization problems based on minimization of the sum squared errors of the objective function residuals. Main advantage of the algorithm is that it…
A numerical method optimizing the coefficients of the semi empirical mass formula or those of similar mass formulas is presented. The optimization is based on the least-squares adjustments method and leads to the resolution of a linear…
Exponential divided differences arise in numerical linear algebra, matrix-function evaluation, and quantum Monte Carlo simulations, where they serve as kernel weights for time evolution and observable estimation. Efficient and numerically…
In this paper we present a convergence rate analysis of inexact variants of several randomized iterative methods. Among the methods studied are: stochastic gradient descent, stochastic Newton, stochastic proximal point and stochastic…
The emergence of huge-scale, data-intensive linear optimization (LO) problems in applications such as machine learning has driven the need for more computationally efficient interior point methods (IPMs). While conventional IPMs are…
The classical method to solve a quadratic optimization problem with nonlinear equality constraints is to solve the Karush-Kuhn-Tucker (KKT) optimality conditions using Newton's method. This approach however is usually computationally…
In this paper, we use the optimization formulation of nonlinear Kalman filtering and smoothing problems to develop second-order variants of iterated Kalman smoother (IKS) methods. We show that Newton's method corresponds to a recursion over…
The algebraic polynomial interpolation on uniformly distributed nodes is affected by the Runge phenomenon, also when the function to be interpolated is analytic. Among all techniques that have been proposed to defeat this phenomenon, there…
In this paper, we study structured quasi-Newton methods for optimization problems with orthogonality constraints. Note that the Riemannian Hessian of the objective function requires both the Euclidean Hessian and the Euclidean gradient. In…
An adaptive regularization strategy for stabilizing Newton-like iterations on a coarse mesh is developed in the context of adaptive finite element methods for nonlinear PDE. Existence, uniqueness and approximation properties are known for…
The constrained mock-Chebyshev least squares operator is a linear approximation operator based on an equispaced grid of points. Like other polynomial or rational approximation methods, it was recently introduced in order to defeat the Runge…
In this paper, we introduce some new iterative optimisation algorithms on Riemannian manifolds and Hilbert spaces which have good global convergence guarantees to local minima. More precisely, these algorithms have the following properties:…
Second-order optimization methods are among the most widely used optimization approaches for convex optimization problems, and have recently been used to optimize non-convex optimization problems such as deep learning models. The widely…
We analyze Newton's method with lazy Hessian updates for solving general possibly non-convex optimization problems. We propose to reuse a previously seen Hessian for several iterations while computing new gradients at each step of the…
In recent years, the proximal gradient method and its variants have been generalized to Riemannian manifolds for solving optimization problems with an additively separable structure, i.e., $f + h$, where $f$ is continuously differentiable,…
Due to their importance in both data analysis and numerical algorithms, low rank approximations have recently been widely studied. They enable the handling of very large matrices. Tight error bounds for the computationally efficient…
Optimization over the Stiefel manifold is a fundamental computational problem in many scientific and engineering applications. Despite considerable research effort, high-dimensional optimization problems over the Stiefel manifold remain…