A Novel Catalyst Scheme for Stochastic Minimax Optimization
Abstract
This paper presents a proximal-point-based catalyst scheme for simple first-order methods applied to convex minimization and convex-concave minimax problems. In particular, for smooth and (strongly)-convex minimization problems, the proposed catalyst scheme, instantiated with a simple variant of stochastic gradient method, attains the optimal rate of convergence in terms of both deterministic and stochastic errors. For smooth and strongly-convex-strongly-concave minimax problems, the catalyst scheme attains the optimal rate of convergence for deterministic and stochastic errors up to a logarithmic factor. To the best of our knowledge, this reported convergence seems to be attained for the first time by stochastic first-order methods in the literature. We obtain this result by designing and catalyzing a novel variant of stochastic extragradient method for solving smooth and strongly-monotone variational inequality, which may be of independent interest.
Cite
@article{arxiv.2311.02814,
title = {A Novel Catalyst Scheme for Stochastic Minimax Optimization},
author = {Guanghui Lan and Yan Li},
journal= {arXiv preprint arXiv:2311.02814},
year = {2023}
}