BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learning
Abstract
There has recently been a surge in research in batch Deep Reinforcement Learning (DRL), which aims for learning a high-performing policy from a given dataset without additional interactions with the environment. We propose a new algorithm, Best-Action Imitation Learning (BAIL), which strives for both simplicity and performance. BAIL learns a V function, uses the V function to select actions it believes to be high-performing, and then uses those actions to train a policy network using imitation learning. For the MuJoCo benchmark, we provide a comprehensive experimental study of BAIL, comparing its performance to four other batch Q-learning and imitation-learning schemes for a large variety of batch datasets. Our experiments show that BAIL's performance is much higher than the other schemes, and is also computationally much faster than the batch Q-learning schemes.
Cite
@article{arxiv.1910.12179,
title = {BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learning},
author = {Xinyue Chen and Zijian Zhou and Zheng Wang and Che Wang and Yanqiu Wu and Keith Ross},
journal= {arXiv preprint arXiv:1910.12179},
year = {2020}
}
Comments
27 pages(15 pages for appendix); Published in 34th Conference on Neural Information Processing Systems (NeurIPS 2020)