Spectral Frank-Wolfe Algorithm: Strict Complementarity and Linear Convergence
Abstract
We develop a novel variant of the classical Frank-Wolfe algorithm, which we call spectral Frank-Wolfe, for convex optimization over a spectrahedron. The spectral Frank-Wolfe algorithm has a novel ingredient: it computes a few eigenvectors of the gradient and solves a small-scale SDP in each iteration. Such procedure overcomes slow convergence of the classical Frank-Wolfe algorithm due to ignoring eigenvalue coalescence. We demonstrate that strict complementarity of the optimization problem is key to proving linear convergence of various algorithms, such as the spectral Frank-Wolfe algorithm as well as the projected gradient method and its accelerated version.
Cite
@article{arxiv.2006.01719,
title = {Spectral Frank-Wolfe Algorithm: Strict Complementarity and Linear Convergence},
author = {Lijun Ding and Yingjie Fei and Qiantong Xu and Chengrun Yang},
journal= {arXiv preprint arXiv:2006.01719},
year = {2020}
}
Comments
The main text has nine pages. Reading the first three sections should give a good understanding of the paper and should take around 15 minutes. Published in proceedings of the 37th International Conference on Machine Learning, online, PMLR 119, 2020