Markov Chain Lifting and Distributed ADMM
Machine Learning
2017-03-14 v1 Data Structures and Algorithms
Information Theory
Machine Learning
math.IT
Optimization and Control
Abstract
The time to converge to the steady state of a finite Markov chain can be greatly reduced by a lifting operation, which creates a new Markov chain on an expanded state space. For a class of quadratic objectives, we show an analogous behavior where a distributed ADMM algorithm can be seen as a lifting of Gradient Descent algorithm. This provides a deep insight for its faster convergence rate under optimal parameter tuning. We conjecture that this gain is always present, as opposed to the lifting of a Markov chain which sometimes only provides a marginal speedup.
Cite
@article{arxiv.1703.03859,
title = {Markov Chain Lifting and Distributed ADMM},
author = {Guilherme França and José Bento},
journal= {arXiv preprint arXiv:1703.03859},
year = {2017}
}
Comments
This work was also selected for a talk at NIPS 2016, Optimization for Machine Learning Workshop (OPT 2016)