Generalization poses a significant challenge in Multi-agent Reinforcement Learning (MARL). The extent to which an agent is influenced by unseen co-players depends on the agent's policy and the specific scenario. A quantitative examination of this relationship sheds light on effectively training agents for diverse scenarios. In this study, we present the Level of Influence (LoI), a metric quantifying the interaction intensity among agents within a given scenario and environment. We observe that, generally, a more diverse set of co-play agents during training enhances the generalization performance of the ego agent; however, this improvement varies across distinct scenarios and environments. LoI proves effective in predicting these improvement disparities within specific scenarios. Furthermore, we introduce a LoI-guided resource allocation method tailored to train a set of policies for diverse scenarios under a constrained budget. Our results demonstrate that strategic resource allocation based on LoI can achieve higher performance than uniform allocation under the same computation budget.
@article{arxiv.2310.07218,
title = {Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalization},
author = {Yuxin Chen and Chen Tang and Ran Tian and Chenran Li and Jinning Li and Masayoshi Tomizuka and Wei Zhan},
journal= {arXiv preprint arXiv:2310.07218},
year = {2023}
}