Momentum-based gradient descent methods for Lie groups
Abstract
Polyak's Heavy Ball (PHB; Polyak, 1964), a.k.a. Classical Momentum, and Nesterov's Accelerated Gradient (NAG; Nesterov, 1983) are well-established momentum-descent methods for optimization. Although the latter generally outperforms the former, primarily, generalizations of PHB-like methods to nonlinear spaces have not been sufficiently explored in the literature. In this paper, we propose a generalization of NAG-like methods for Lie group optimization. This generalization is based on the variational one-to-one correspondence between classical and accelerated momentum methods (Campos et al., 2023). We provide numerical experiments for chosen retractions on the group of rotations based on the Frobenius norm and the Rosenbrock function to demonstrate the effectiveness of our proposed methods, and that align with results of the Euclidean case, that is, a faster convergence rate for NAG.
Keywords
Cite
@article{arxiv.2404.09363,
title = {Momentum-based gradient descent methods for Lie groups},
author = {Cédric M. Campos and David Martín de Diego and José Torrente},
journal= {arXiv preprint arXiv:2404.09363},
year = {2025}
}
Comments
22 pages, 2 algorithms, 6 figures