Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents
Abstract
There has been rapidly growing interest in meta-learning as a method for increasing the flexibility and sample efficiency of reinforcement learning. One problem in this area of research, however, has been a scarcity of adequate benchmark tasks. In general, the structure underlying past benchmarks has either been too simple to be inherently interesting, or too ill-defined to support principled analysis. In the present work, we introduce a new benchmark for meta-RL research, emphasizing transparency and potential for in-depth analysis as well as structural richness. Alchemy is a 3D video game, implemented in Unity, which involves a latent causal structure that is resampled procedurally from episode to episode, affording structure learning, online inference, hypothesis testing and action sequencing based on abstract domain knowledge. We evaluate a pair of powerful RL agents on Alchemy and present an in-depth analysis of one of these agents. Results clearly indicate a frank and specific failure of meta-learning, providing validation for Alchemy as a challenging benchmark for meta-RL. Concurrent with this report, we are releasing Alchemy as public resource, together with a suite of analysis tools and sample agent trajectories.
Keywords
Cite
@article{arxiv.2102.02926,
title = {Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents},
author = {Jane X. Wang and Michael King and Nicolas Porcel and Zeb Kurth-Nelson and Tina Zhu and Charlie Deck and Peter Choy and Mary Cassin and Malcolm Reynolds and Francis Song and Gavin Buttimore and David P. Reichert and Neil Rabinowitz and Loic Matthey and Demis Hassabis and Alexander Lerchner and Matthew Botvinick},
journal= {arXiv preprint arXiv:2102.02926},
year = {2021}
}
Comments
Published in Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 2021