Primal-Dual Sequential Subspace Optimization for Saddle-point Problems
Optimization and Control
2020-08-24 v1 Machine Learning
Machine Learning
Abstract
We introduce a new sequential subspace optimization method for large-scale saddle-point problems. It solves iteratively a sequence of auxiliary saddle-point problems in low-dimensional subspaces, spanned by directions derived from first-order information over the primal \emph{and} dual variables. Proximal regularization is further deployed to stabilize the optimization process. Experimental results demonstrate significantly better convergence relative to popular first-order methods. We analyze the influence of the subspace on the convergence of the algorithm, and assess its performance in various deterministic optimization scenarios, such as bi-linear games, ADMM-based constrained optimization and generative adversarial networks.
Cite
@article{arxiv.2008.09149,
title = {Primal-Dual Sequential Subspace Optimization for Saddle-point Problems},
author = {Yoni Choukroun and Michael Zibulevsky and Pavel Kisilev},
journal= {arXiv preprint arXiv:2008.09149},
year = {2020}
}