Learning Relative Return Policies With Upside-Down Reinforcement Learning
Abstract
Lately, there has been a resurgence of interest in using supervised learning to solve reinforcement learning problems. Recent work in this area has largely focused on learning command-conditioned policies. We investigate the potential of one such method -- upside-down reinforcement learning -- to work with commands that specify a desired relationship between some scalar value and the observed return. We show that upside-down reinforcement learning can learn to carry out such commands online in a tabular bandit setting and in CartPole with non-linear function approximation. By doing so, we demonstrate the power of this family of methods and open the way for their practical use under more complicated command structures.
Cite
@article{arxiv.2202.12742,
title = {Learning Relative Return Policies With Upside-Down Reinforcement Learning},
author = {Dylan R. Ashley and Kai Arulkumaran and Jürgen Schmidhuber and Rupesh Kumar Srivastava},
journal= {arXiv preprint arXiv:2202.12742},
year = {2022}
}
Comments
presented at the 5th Multidisciplinary Conference on Reinforcement Learning and Decision Making; 5 pages in main text, 2 figures in main text