Compositional generalization is a critical ability in learning and decision-making. We focus on the setting of reinforcement learning in object-oriented environments to study compositional generalization in world modeling. We (1) formalize the compositional generalization problem with an algebraic approach and (2) study how a world model can achieve that. We introduce a conceptual environment, Object Library, and two instances, and deploy a principled pipeline to measure the generalization ability. Motivated by the formulation, we analyze several methods with exact or no compositional generalization ability using our framework, and design a differentiable approach, Homomorphic Object-oriented World Model (HOWM), that achieves soft but more efficient compositional generalization.
@article{arxiv.2204.13661,
title = {Toward Compositional Generalization in Object-Oriented World Modeling},
author = {Linfeng Zhao and Lingzhi Kong and Robin Walters and Lawson L. S. Wong},
journal= {arXiv preprint arXiv:2204.13661},
year = {2022}
}
Comments
ICML 2022 Long Presentation. Website: http://lfzhao.com/oowm/