The objective of lifelong reinforcement learning (RL) is to optimize agents which can continuously adapt and interact in changing environments. However, current RL approaches fail drastically when environments are non-stationary and interactions are non-episodic. We propose Lifelong Skill Planning (LiSP), an algorithmic framework for non-episodic lifelong RL based on planning in an abstract space of higher-order skills. We learn the skills in an unsupervised manner using intrinsic rewards and plan over the learned skills using a learned dynamics model. Moreover, our framework permits skill discovery even from offline data, thereby reducing the need for excessive real-world interactions. We demonstrate empirically that LiSP successfully enables long-horizon planning and learns agents that can avoid catastrophic failures even in challenging non-stationary and non-episodic environments derived from gridworld and MuJoCo benchmarks.
@article{arxiv.2012.03548,
title = {Reset-Free Lifelong Learning with Skill-Space Planning},
author = {Kevin Lu and Aditya Grover and Pieter Abbeel and Igor Mordatch},
journal= {arXiv preprint arXiv:2012.03548},
year = {2021}
}
Comments
In the proceedings of the 7th International Conference on Learning Representations (ICLR), Virtual, April 2021