Related papers: Optimization on manifolds: A symplectic approach
We introduce a recent symplectic integration scheme derived for solving physically motivated systems with non-separable Hamiltonians. We show its relevance to Riemannian manifold Hamiltonian Monte Carlo (RMHMC) and provide an alternative to…
Low-rank optimization problems with sparse simplex constraints involve variables that must satisfy nonnegativity, sparsity, and sum-to-1 conditions, making their optimization particularly challenging due to the interplay between low-rank…
We study the discretization of (almost-)Dirac structures using the notion of retraction and discretization maps on manifolds. Additionally, we apply the proposed discretization techniques to obtain numerical integrators for port-Hamiltonian…
Arguably, the two most popular accelerated or momentum-based optimization methods in machine learning are Nesterov's accelerated gradient and Polyaks's heavy ball, both corresponding to different discretizations of a particular second order…
In order to perform numerical studies of long-term stability in nonlinear Hamiltonian systems, one needs a numerical integration algorithm which is symplectic. Further, this algorithm should be fast and accurate. In this paper, we propose…
This paper proposes a general framework of Riemannian adaptive optimization methods. The framework encapsulates several stochastic optimization algorithms on Riemannian manifolds and incorporates the mini-batch strategy that is often used…
Optimization with orthogonality constraints frequently arises in various fields such as machine learning. Riemannian optimization offers a powerful framework for solving these problems by equipping the constraint set with a Riemannian…
The techniques and analysis presented in this thesis provide new methods to solve optimization problems posed on Riemannian manifolds. These methods are applied to the subspace tracking problem found in adaptive signal processing and…
Most numerical integration algorithms are not designed specifically for Hamiltonian systems and do not respect their characteristic properties, which include the preservation of phase space volume with time. This can lead to spurious…
Hamiltonian Monte Carlo (HMC) algorithms which combine numerical approximation of Hamiltonian dynamics on finite intervals with stochastic refreshment and Metropolis correction are popular sampling schemes, but it is known that they may…
We consider optimization problems on manifolds with equality and inequality constraints. A large body of work treats constrained optimization in Euclidean spaces. In this work, we consider extensions of existing algorithms from the…
In this paper, explicit stable integrators based on symplectic and contact geometries are proposed for a non-autonomous ordinarily differential equation (ODE) found in improving convergence rate of Nesterov's accelerated gradient method.…
This paper considers a stochastic optimization problem over the fixed point sets of quasinonexpansive mappings on Riemannian manifolds. The problem enables us to consider Riemannian hierarchical optimization problems over complicated sets,…
We compare the performances of symplectic and non-symplectic integrators for the computation of normal geodesics and conjugate points in sub-Riemannian geometry at the example of the Martinet case. For this case study we consider first the…
We present and analyze a framework for designing symplectic neural networks (SympNets) based on geometric integrators for Hamiltonian differential equations. The SympNets are universal approximators in the space of Hamiltonian…
Symplectic integrators for Hamiltonian systems have been quite successful for studying few-body dynamical systems. These integrators are frequently derived using a formalism built on symplectic maps. There have been recent efforts to extend…
Evolutionary partial differential equations play a crucial role in many areas of science and engineering. Spatial discretization of these equations leads to a system of ordinary differential equations which can then be solved by numerical…
This paper aims to investigate the distributed stochastic optimization problems on compact embedded submanifolds (in the Euclidean space) for multi-agent network systems. To address the manifold structure, we propose a distributed…
In recent years, interest in gradient-based optimization over Riemannian manifolds has surged. However, a significant challenge lies in the reliance on hyperparameters, especially the learning rate, which requires meticulous tuning by…
Symplectic integration of autonomous Hamiltonian systems is a well-known field of study in geometric numerical integration, but for non-autonomous systems the situation is less clear, since symplectic structure requires an even number of…