We study computationally efficient methods for finding equilibria in n-player general-sum games, specifically ones that afford complex visuomotor skills. We show how existing methods would struggle in this setting, either computationally or in theory. We then introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Correlated Equilibrium (CCE) of the game. We show empirical convergence in a suite of OpenSpiel games, validated rigorously by exact game solvers. We then deploy NeuPL-JPSRO to complex domains, where our approach enables adaptive coordination in a MuJoCo control domain and skill transfer in capture-the-flag. Our work shows that equilibrium convergent population learning can be implemented at scale and in generality, paving the way towards solving real-world games between heterogeneous players with mixed motives.
@article{arxiv.2401.05133,
title = {Neural Population Learning beyond Symmetric Zero-sum Games},
author = {Siqi Liu and Luke Marris and Marc Lanctot and Georgios Piliouras and Joel Z. Leibo and Nicolas Heess},
journal= {arXiv preprint arXiv:2401.05133},
year = {2024}
}