Related papers: Generalization of Zeroth-Order Method for Quotient…
The maximization of the (generalized) Rayleigh quotient is a central problem in numerical linear algebra. Conventional algorithms for its computation typically rely on matrix-adjoint products, making them sensitive to errors arising from…
The techniques and analysis presented in this paper provide new methods to solve optimization problems posed on Riemannian manifolds. A new point of view is offered for the solution of constrained optimization problems. Some classical…
This paper considers the problem of computing the operator norm of a linear map between finite dimensional Hilbert spaces when only evaluations of the linear map are available and under restrictive storage assumptions. We propose a…
We introduce a generalized Rayleigh-quotient on the tensor product of Grassmannians enabling a unified approach to well-known optimization tasks from different areas of numerical linear algebra, such as best low-rank approximations of…
This paper introduces an efficient algorithm for computing the general oscillatory matrix functions. These computations are crucial for solving second-order semi-linear initial value problems. The method is exploited using the scaling and…
Rayleigh quotient minimization deals with optimizing a quadratic homogeneous function over a sphere. Its critical points correspond to the normalized eigenvectors of the symmetric matrix associated with the quadratic form. In this paper, we…
We give quantum speedups of several general-purpose numerical optimisation methods for minimising a function $f:\mathbb{R}^n \to \mathbb{R}$. First, we show that many techniques for global optimisation under a Lipschitz constraint can be…
Maximizing the sum of two generalized Rayleigh quotients (SRQ) can be reformulated as a one-dimensional optimization problem, where the function value evaluations are reduced to solving semi-definite programming (SDP) subproblems. In this…
Gradient descent method, as one of the major methods in numerical optimization, is the key ingredient in many machine learning algorithms. As one of the most fundamental way to solve the optimization problems, it promises the function value…
We generalize the Rayleigh Quotient Iteration (RQI) to the problem of solving a nonlinear equation where the variables are divided into two subsets, one satisfying additional equality constraints and the other could be considered as…
This paper addresses the global optimization of the sum of the Rayleigh quotient and the generalized Rayleigh quotient on the unit sphere. While various methods have been proposed for this problem, they fail to reliably converge to the…
Optimization problems with access to only zeroth-order information of the objective function on Riemannian manifolds arise in various applications, spanning from statistical learning to robot learning. While various zeroth-order algorithms…
Successive quadratic approximations, or second-order proximal methods, are useful for minimizing functions that are a sum of a smooth part and a convex, possibly nonsmooth part that promotes regularization. Most analyses of iteration…
Quasar-convex functions form a broad nonconvex class with applications to linear dynamical systems, generalized linear models, and Riemannian optimization, among others. Current nearly optimal algorithms work only in affine spaces due to…
Optimization problems in disciplines such as machine learning are commonly solved with iterative methods. Gradient descent algorithms find local minima by moving along the direction of steepest descent while Newton's method takes into…
We consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in Euclidean space, where the task is to solve Riemannian optimization problem with only noisy objective function evaluations. Towards this, our main…
This manuscript presents a novel and reliable third-order iterative procedure for computing the zeros of solutions to second-order ordinary differential equations. By approximating the solution of the related Riccati differential equation…
Gradient-based algorithms, popular strategies to optimization problems, are essential for many modern machine-learning techniques. Theoretically, extreme points of certain cost functions can be found iteratively along the directions of the…
Zeroth-order optimization addresses problems where gradient information is inaccessible or impractical to compute. While most existing methods rely on first-order approximations, incorporating second-order (curvature) information can, in…
We propose a stochastic variance-reduced cubic regularized Newton algorithm to optimize the finite-sum problem over a Riemannian submanifold of the Euclidean space. The proposed algorithm requires a full gradient and Hessian update at the…