We show how an ensemble of Q∗-functions can be leveraged for more effective exploration in deep reinforcement learning. We build on well established algorithms from the bandit setting, and adapt them to the Q-learning setting. We propose an exploration strategy based on upper-confidence bounds (UCB). Our experiments show significant gains on the Atari benchmark.
Cite
@article{arxiv.1706.01502,
title = {UCB Exploration via Q-Ensembles},
author = {Richard Y. Chen and Szymon Sidor and Pieter Abbeel and John Schulman},
journal= {arXiv preprint arXiv:1706.01502},
year = {2017}
}