A Matrix Splitting Perspective on Planning with Options
Abstract
We show that the Bellman operator underlying the options framework leads to a matrix splitting, an approach traditionally used to speed up convergence of iterative solvers for large linear systems of equations. Based on standard comparison theorems for matrix splittings, we then show how the asymptotic rate of convergence varies as a function of the inherent timescales of the options. This new perspective highlights a trade-off between asymptotic performance and the cost of computation associated with building a good set of options.
Cite
@article{arxiv.1612.00916,
title = {A Matrix Splitting Perspective on Planning with Options},
author = {Pierre-Luc Bacon and Doina Precup},
journal= {arXiv preprint arXiv:1612.00916},
year = {2017}
}
Comments
The results presented in the previous version of this paper were found be applicable only to "gating execution" and not "call-and-return". We made this distinction clear in the text and added an extension to the call-and-return model