IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL
Machine Learning
2021-01-05 v5 Machine Learning
Abstract
We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment -- the ability to reliably reach a diverse set of states and show how to identify sub-goals as states with high necessary option information through an information theoretic regularizer. Despite being discovered without explicit goal supervision, our sub-goals provide better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised counterparts in prior work.
Keywords
Cite
@article{arxiv.1907.10580,
title = {IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL},
author = {Nirbhay Modhe and Prithvijit Chattopadhyay and Mohit Sharma and Abhishek Das and Devi Parikh and Dhruv Batra and Ramakrishna Vedantam},
journal= {arXiv preprint arXiv:1907.10580},
year = {2021}
}