Reinforcement learning (RL) algorithms involve the deep nesting of highly irregular computation patterns, each of which typically exhibits opportunities for distributed computation. We argue for distributing RL components in a composable way by adapting algorithms for top-down hierarchical control, thereby encapsulating parallelism and resource requirements within short-running compute tasks. We demonstrate the benefits of this principle through RLlib: a library that provides scalable software primitives for RL. These primitives enable a broad range of algorithms to be implemented with high performance, scalability, and substantial code reuse. RLlib is available at https://rllib.io/.
@article{arxiv.1712.09381,
title = {RLlib: Abstractions for Distributed Reinforcement Learning},
author = {Eric Liang and Richard Liaw and Philipp Moritz and Robert Nishihara and Roy Fox and Ken Goldberg and Joseph E. Gonzalez and Michael I. Jordan and Ion Stoica},
journal= {arXiv preprint arXiv:1712.09381},
year = {2018}
}
Comments
Published in the International Conference on Machine Learning (ICML 2018), 10 pages