Momentum-based gradient descent methods for Lie groups

Cédric M. Campos, David Martín de Diego, José Torrente
4/14/2024
math.OCcs.LGmath.DG

No official code found. Best implementation: cmcampos-xyz/paper-2021-momentum-methods-for-lie-groups (1 star, MIT, last commit 2025-07-31).

Abstract

Polyak's Heavy Ball (PHB; Polyak, 1964), a.k.a. Classical Momentum, and Nesterov's Accelerated Gradient (NAG; Nesterov, 1983) are well-established momentum-descent methods for optimization. Although the latter generally outperforms the former, primarily, generalizations of PHB-like methods to nonlinear spaces have not been sufficiently explored in the literature. In this paper, we propose a generalization of NAG-like methods for Lie group optimization. This generalization is based on the variational one-to-one correspondence between classical and accelerated momentum methods (Campos et al., 2023). We provide numerical experiments for chosen retractions on the group of rotations based on the Frobenius norm and the Rosenbrock function to demonstrate the effectiveness of our proposed methods, and that align with results of the Euclidean case, that is, a faster convergence rate for NAG.

View on arXivView PDF

Code Implementations(1)

cmcampos-xyz/paper-2021-momentum-methods-for-lie-groupsCommunity68%
MIT

Momentum-based gradient descent methods for Lie groups

10JuliaMay 12, 20241 years agoMIT

Similar Papers

Quantitative Convergences of Lie Group Momentum Optimizers

89% match1 repo2024

Modified Loss of Momentum Gradient Descent: Fine-Grained Analysis

82% match2025

Robust Gradient Descent via Heavy-Ball Momentum with Predictive Extrapolation

80% match2025

Convergence of Kinetic Langevin Monte Carlo on Lie groups

80% match2024

Convergence of Momentum-Based Optimization Algorithms with Time-Varying Parameters

78% match2025

Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance

77% match1 repo12024

Cite this paper

@article{campos2024momentumbased,
  title  = {Momentum-based gradient descent methods for Lie groups},
  author = {Cédric M. Campos and David Martín de Diego and José Torrente},
  year   = {2024},
  eprint = {2404.09363},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.09363v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.09363.svg)](https://codeofpaper.com/paper/2404.09363)

Discussion