Semi-On-Policy Training for Sample Efficient Multi-Agent Policy Gradients
Abstract
Policy gradient methods are an attractive approach to multi-agent reinforcement learning problems due to their convergence properties and robustness in partially observable scenarios. However, there is a significant performance gap between state-of-the-art policy gradient and value-based methods on the popular StarCraft Multi-Agent Challenge (SMAC) benchmark. In this paper, we introduce semi-on-policy (SOP) training as an effective and computationally efficient way to address the sample inefficiency of on-policy policy gradient methods. We enhance two state-of-the-art policy gradient algorithms with SOP training, demonstrating significant performance improvements. Furthermore, we show that our methods perform as well or better than state-of-the-art value-based methods on a variety of SMAC tasks.
Cite
@article{arxiv.2104.13446,
title = {Semi-On-Policy Training for Sample Efficient Multi-Agent Policy Gradients},
author = {Bozhidar Vasilev and Tarun Gupta and Bei Peng and Shimon Whiteson},
journal= {arXiv preprint arXiv:2104.13446},
year = {2021}
}
Comments
AAMAS Adaptive and Learning Agents Workshop. 20th International Conference on Autonomous Agents and Multiagent Systems