Model-Based Regularization for Deep Reinforcement Learning with Transcoder Networks
Abstract
This paper proposes a new optimization objective for value-based deep reinforcement learning. We extend conventional Deep Q-Networks (DQNs) by adding a model-learning component yielding a transcoder network. The prediction errors for the model are included in the basic DQN loss as additional regularizers. This augmented objective leads to a richer training signal that provides feedback at every time step. Moreover, because learning an environment model shares a common structure with the RL problem, we hypothesize that the resulting objective improves both sample efficiency and performance. We empirically confirm our hypothesis on a range of 20 games from the Atari benchmark attaining superior results over vanilla DQN without model-based regularization.
Cite
@article{arxiv.1809.01906,
title = {Model-Based Regularization for Deep Reinforcement Learning with Transcoder Networks},
author = {Felix Leibfried and Peter Vrancx},
journal= {arXiv preprint arXiv:1809.01906},
year = {2018}
}
Comments
Presented at the NIPS Deep Reinforcement Learning Workshop, Montreal, Canada, 2018