Combining Model-Free Q-Ensembles and Model-Based Approaches for Informed Exploration
Abstract
Q-Ensembles are a model-free approach where input images are fed into different Q-networks and exploration is driven by the assumption that uncertainty is proportional to the variance of the output Q-values obtained. They have been shown to perform relatively well compared to other exploration strategies. Further, model-based approaches, such as encoder-decoder models have been used successfully for next frame prediction given previous frames. This paper proposes to integrate the model-free Q-ensembles and model-based approaches with the hope of compounding the benefits of both and achieving superior exploration as a result. Results show that a model-based trajectory memory approach when combined with Q-ensembles produces superior performance when compared to only using Q-ensembles.
Cite
@article{arxiv.1806.04552,
title = {Combining Model-Free Q-Ensembles and Model-Based Approaches for Informed Exploration},
author = {Sreecharan Sankaranarayanan and Raghuram Mandyam Annasamy and Katia Sycara and Carolyn Penstein Rosé},
journal= {arXiv preprint arXiv:1806.04552},
year = {2018}
}
Comments
Submitted to the Thirty-Second Annual Conference on Neural Information Processing Systems (NIPS 2018)