Stochastic Primal-Dual Method for Empirical Risk Minimization with $\mathcal{O}(1)$ Per-Iteration Complexity
Optimization and Control
2018-11-06 v1 Machine Learning
Abstract
Regularized empirical risk minimization problem with linear predictor appears frequently in machine learning. In this paper, we propose a new stochastic primal-dual method to solve this class of problems. Different from existing methods, our proposed methods only require O(1) operations in each iteration. We also develop a variance-reduction variant of the algorithm that converges linearly. Numerical experiments suggest that our methods are faster than existing ones such as proximal SGD, SVRG and SAGA on high-dimensional problems.
Cite
@article{arxiv.1811.01182,
title = {Stochastic Primal-Dual Method for Empirical Risk Minimization with $\mathcal{O}(1)$ Per-Iteration Complexity},
author = {Conghui Tan and Tong Zhang and Shiqian Ma and Ji Liu},
journal= {arXiv preprint arXiv:1811.01182},
year = {2018}
}
Comments
NIPS 2018