Related papers: Optimization Methods on Riemannian Manifolds via E…

Orthogonality constraints naturally appear in many machine learning problems, from principal component analysis to robust neural network training. They are usually solved using Riemannian optimization algorithms, which minimize the…

This paper studies large-scale optimization problems on Riemannian manifolds whose objective function is a finite sum of negative log-probability losses. Such problems arise in various machine learning and signal processing applications. By…

In this work, we analyze two of the most fundamental algorithms in geodesically convex optimization: Riemannian gradient descent and (possibly inexact) Riemannian proximal point. We quantify their rates of convergence and produce different…

A recently introduced general-purpose heuristic for finding high-quality solutions for many hard optimization problems is reviewed. The method is inspired by recent progress in understanding far-from-equilibrium phenomena in terms of {\em…

Variance parameter estimation in linear mixed models is a challenge for many classical nonlinear optimization algorithms due to the positive-definiteness constraint of the random effects covariance matrix. We take a completely novel view on…

We propose a new design strategy for extremum seeking control for a multi-dimensional single-integrator system in the presence of local extrema. The proposed method employs suitably designed sinusoidal dither signals, which force the…

In this paper, we introduce some new iterative optimisation algorithms on Riemannian manifolds and Hilbert spaces which have good global convergence guarantees to local minima. More precisely, these algorithms have the following properties:…

Large-scale optimization problems arising from the discretization of problems involving PDEs sometimes admit solutions that can be well approximated by low-rank matrices. In this paper, we will exploit this low-rank approximation property…

Many modern machine learning applications - from online principal component analysis to covariance matrix identification and dictionary learning - can be formulated as minimization problems on Riemannian manifolds, and are typically solved…

We study smooth stochastic optimization problems on Riemannian manifolds. Via adapting the recently proposed SPIDER algorithm \citep{fang2018spider} (a variance reduced stochastic method) to Riemannian manifold, we can achieve faster rate…

Extremum seeking systems are powerful methods able to steer the input of a (dynamical) cost function towards an optimizer, without any prior knowledge of the cost function. To achieve their objective, they typically combine time-periodic…

Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where…

We introduce a Riemannian optimistic online learning algorithm for Hadamard manifolds based on inexact implicit updates. Unlike prior work, our method can handle in-manifold constraints, and matches the best known regret bounds in the…

This paper presents a class of efficient manifold optimization algorithms for computing the ground state solutions of a semilinear elliptic system, which are unstable saddle points of the variational functional. Variational arguments show…

This paper considers the optimization problem in the form of $\min_{X \in \mathcal{F}_v} f(x) + \lambda \|X\|_1,$ where $f$ is smooth, $\mathcal{F}_v = \{X \in \mathbb{R}^{n \times q} : X^T X = I_q, v \in \mathrm{span}(X)\}$, and $v$ is a…

We consider the optimization problem with a generally quadratic matrix constraint of the form $X^TAX = J$, where $A$ is a given nonsingular, symmetric $n\times n$ matrix and $J$ is a given $k\times k$ symmetric matrix, with $k\leq n$,…

This paper proposes a novel general framework of Riemannian conjugate gradient methods, that is, conjugate gradient methods on Riemannian manifolds. The conjugate gradient methods are important first-order optimization algorithms both in…

Optimization tasks are crucial in statistical machine learning. Recently, there has been great interest in leveraging tools from dynamical systems to derive accelerated and robust optimization methods via suitable discretizations of…

Bilevel optimization has gained prominence in various applications. In this study, we introduce a framework for solving bilevel optimization problems, where the variables in both the lower and upper levels are constrained on Riemannian…

We consider discretized two-dimensional PDE-constrained shape optimization problems, in which shapes are represented by triangular meshes. Given the connectivity, the space of admissible vertex positions was recently identified to be a…