Convergence of two-timescale gradient descent ascent dynamics: finite-dimensional and mean-field perspectives
Optimization and Control
2025-10-13 v3 Machine Learning
Numerical Analysis
Numerical Analysis
Abstract
The two-timescale gradient descent-ascent (GDA) is a canonical gradient algorithm designed to find Nash equilibria in min-max games. We analyze the two-timescale GDA by investigating the effects of learning rate ratios on convergence behavior in both finite-dimensional and mean-field settings. In particular, for finite-dimensional quadratic min-max games, we obtain long-time convergence in near quasi-static regimes through the hypocoercivity method. For mean-field GDA dynamics, we investigate convergence under a finite-scale ratio using a mixed synchronous-reflection coupling technique.
Keywords
Cite
@article{arxiv.2501.17122,
title = {Convergence of two-timescale gradient descent ascent dynamics: finite-dimensional and mean-field perspectives},
author = {Jing An and Jianfeng Lu},
journal= {arXiv preprint arXiv:2501.17122},
year = {2025}
}
Comments
v3: added analysis of preconditioned GDA for finite-dimensional case and improved results for mean-field case