Proximal Gradient methods with Adaptive Subspace Sampling
Optimization and Control
2020-04-29 v1
Abstract
Many applications in machine learning or signal processing involve nonsmooth optimization problems. This nonsmoothness brings a low-dimensional structure to the optimal solutions. In this paper, we propose a randomized proximal gradient method harnessing this underlying structure. We introduce two key components: i) a random subspace proximal gradient algorithm; ii) an identification-based sampling of the subspaces. Their interplay brings a significant performance improvement on typical learning problems in terms of dimensions explored.
Cite
@article{arxiv.2004.13356,
title = {Proximal Gradient methods with Adaptive Subspace Sampling},
author = {Dmitry Grishchenko and Franck Iutzeler and Jérôme Malick},
journal= {arXiv preprint arXiv:2004.13356},
year = {2020}
}