We propose a framework that leverages foundation models as teachers, guiding a reinforcement learning agent to acquire semantically meaningful behavior without human feedback. In our framework, the agent receives task instructions grounded in a training environment from large language models. Then, a vision-language model guides the agent in learning the multi-task language-conditioned policy by providing reward feedback. We demonstrate that our method can learn semantically meaningful skills in a challenging open-ended MineDojo environment while prior unsupervised skill discovery methods struggle. Additionally, we discuss observed challenges of using off-the-shelf foundation models as teachers and our efforts to address them.
@article{arxiv.2312.08958,
title = {LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers},
author = {Taewook Nam and Juyong Lee and Jesse Zhang and Sung Ju Hwang and Joseph J. Lim and Karl Pertsch},
journal= {arXiv preprint arXiv:2312.08958},
year = {2023}
}
Comments
2nd Workshop on Agent Learning in Open-Endedness (ALOE) at NeurIPS 2023