Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems
Machine Learning
2014-07-11 v1 Machine Learning
Abstract
Recent advances in optimization theory have shown that smooth strongly convex finite sums can be minimized faster than by treating them as a black box "batch" problem. In this work we introduce a new method in this class with a theoretical convergence rate four times faster than existing methods, for sums with sufficiently many terms. This method is also amendable to a sampling without replacement scheme that in practice gives further speed-ups. We give empirical results showing state of the art performance.
Cite
@article{arxiv.1407.2710,
title = {Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems},
author = {Aaron J. Defazio and Tibério S. Caetano and Justin Domke},
journal= {arXiv preprint arXiv:1407.2710},
year = {2014}
}