Reinforcement learning competitions advance the field by providing appropriate scope and support to develop solutions toward a specific problem. To promote the development of more broadly applicable methods, organizers need to enforce the use of general techniques, the use of sample-efficient methods, and the reproducibility of the results. While beneficial for the research community, these restrictions come at a cost -- increased difficulty. If the barrier for entry is too high, many potential participants are demoralized. With this in mind, we hosted the third edition of the MineRL ObtainDiamond competition, MineRL Diamond 2021, with a separate track in which we permitted any solution to promote the participation of newcomers. With this track and more extensive tutorials and support, we saw an increased number of submissions. The participants of this easier track were able to obtain a diamond, and the participants of the harder track progressed the generalizable solutions in the same task.
@article{arxiv.2202.10583,
title = {MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned},
author = {Anssi Kanervisto and Stephanie Milani and Karolis Ramanauskas and Nicholay Topin and Zichuan Lin and Junyou Li and Jianing Shi and Deheng Ye and Qiang Fu and Wei Yang and Weijun Hong and Zhongyue Huang and Haicheng Chen and Guangjun Zeng and Yue Lin and Vincent Micheli and Eloi Alonso and François Fleuret and Alexander Nikulin and Yury Belousov and Oleg Svidchenko and Aleksei Shpilman},
journal= {arXiv preprint arXiv:2202.10583},
year = {2022}
}
Comments
Under review for PMLR volume on NeurIPS 2021 competitions