Related papers: Optimization flows landing on the Stiefel manifold
In this paper, we consider a class of optimization problems constrained to the generalized Stiefel manifold. Such problems are fundamental to a wide range of real-world applications, including generalized canonical correlation analysis,…
We consider the optimization problem with a generally quadratic matrix constraint of the form $X^TAX = J$, where $A$ is a given nonsingular, symmetric $n\times n$ matrix and $J$ is a given $k\times k$ symmetric matrix, with $k\leq n$,…
Bilevel optimization is a key framework in hierarchical decision-making, where one problem is embedded within the constraints of another. In this work, we propose a control-theoretic approach to solving bilevel optimization problems. Our…
This paper studies minimax optimization problems defined over infinite-dimensional function classes of overparameterized two-layer neural networks. In particular, we consider the minimax optimization problem stemming from estimating linear…
This paper proposes a theoretical framework for modeling and optimizing the bounded functions based on the Fourier series approximation and Ricci flow. Specifically, the initial manifold, $\mathcal{M}_0$ is approximated using Fourier series…
Flow matching has emerged as a powerful framework for generative modeling through continuous normalizing flows. We investigate a potential topological constraint: when the prior distribution and target distribution have mismatched topology…
Many problems in machine learning can be formulated as solving entropy-regularized optimal transport on the space of probability measures. The canonical approach involves the Sinkhorn iterates, renowned for their rich mathematical…
Recently, optimization on the Riemannian manifold have provided valuable insights to the optimization community. In this regard, extending these methods to to the Wasserstein space is of particular interest, since optimization on…
We consider the consensual distributed optimization problem in the Riemannian context. Specifically, the minimization of a sum of functions form is studied where each individual function in the sum is located at the node of a network. An…
When considering a general system of equations describing the space-time evolution (flow) of one or several variables, the problem of the optimization over a finite period of time of a measure of the state variable at the final time is a…
This paper proposes novel gradient-flow schemes that yield convergence to the optimal point of a convex optimization problem within a \textit{fixed} time from any given initial condition for unconstrained optimization, constrained…
Accelerated gradient methods are the cornerstones of large-scale, data-driven optimization problems that arise naturally in machine learning and other fields concerning data analysis. We introduce a gradient-based optimization framework for…
Optimization over the Stiefel manifold is a fundamental computational problem in many scientific and engineering applications. Despite considerable research effort, high-dimensional optimization problems over the Stiefel manifold remain…
In this work we propose a method to perform optimization on manifolds. We assume to have an objective function $f$ defined on a manifold and think of it as the potential energy of a mechanical system. By adding a momentum-dependent kinetic…
We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this kind of problems can be classified into three classes.…
This article presents a novel resolution to the problem of spline interpolation versus least-squares fitting on smooth Riemannian manifolds utilizing the method of gradient flows of networks. This approach represents a contribution to both…
Normalizing flows have shown great promise for modelling flexible probability distributions in a computationally tractable way. However, whilst data is often naturally described on Riemannian manifolds such as spheres, torii, and hyperbolic…
Optimization problems on the generalized Stiefel manifold (and products of it) are prevalent across science and engineering. For example, in computational science they arise in symmetric (generalized) eigenvalue problems, in nonlinear…
Optimization tasks are crucial in statistical machine learning. Recently, there has been great interest in leveraging tools from dynamical systems to derive accelerated and robust optimization methods via suitable discretizations of…
Traditional theories of optimization cannot describe the dynamics of optimization in deep learning, even in the simple setting of deterministic training. The challenge is that optimizers typically operate in a complex, oscillatory regime…