Related papers: Provable Fictitious Play for General Mean-Field Ga…
In this paper, we propose a passivity-based methodology for analysis and design of reinforcement learning in multi-agent finite games. Starting from a known exponentially-discounted reinforcement learning scheme, we show that convergence to…
Mean field games are studied by means of the weak formulation of stochastic optimal control. This approach allows the mean field interactions to enter through both state and control processes and take a form which is general enough to…
In this paper, we introduce a natural learning rule for mean field games with finite state and action space, the so-called myopic adjustment process. The main motivation for these considerations are the complex computations necessary to…
Recently, the paper [12] introduces a derivative-free consensus-based particle method that finds the Nash equilibrium of non-convex multiplayer games, where it proves the global exponential convergence in the sense of mean-field law. This…
We explore the use of policy approximations to reduce the computational cost of learning Nash equilibria in zero-sum stochastic games. We propose a new Q-learning type algorithm that uses a sequence of entropy-regularized soft policies to…
We present a novel variant of fictitious play dynamics combining classical fictitious play with Q-learning for stochastic games and analyze its convergence properties in two-player zero-sum stochastic games. Our dynamics involves players…
Despite the notable successes in video games such as Atari 2600, current AI is yet to defeat human champions in the domain of real-time strategy (RTS) games. One of the reasons is that an RTS game is a multi-agent game, in which…
We study mean field games and corresponding $N$-player games in continuous time over a finite time horizon where the position of each agent belongs to a finite state space. As opposed to previous works on finite state mean field games, we…
In this paper, we study finite-agent linear-quadratic games on graphs. Specifically, we propose a comprehensive framework that extends the existing literature by incorporating heterogeneous and interpretable player interactions. Compared to…
Multi-agent reinforcement learning is a challenging and active field of research due to the inherent nonstationary property and coupling between agents. A popular approach to modeling the multi-agent interactions underlying the multi-agent…
Fictitious play is a popular learning algorithm in which players that utilize the history of actions played by the players and the knowledge of their own payoff matrix can converge to the Nash equilibrium under certain conditions on the…
In this paper we present a scalable deep learning framework for finding Markovian Nash Equilibria in multi-agent stochastic games using fictitious play. The motivation is inspired by theoretical analysis of Forward Backward Stochastic…
We study the global convergence of policy optimization for finding the Nash equilibria (NE) in zero-sum linear quadratic (LQ) games. To this end, we first investigate the landscape of LQ games, viewing it as a nonconvex-nonconcave…
This paper considers the problem of inverse reinforcement learning in zero-sum stochastic games when expert demonstrations are known to be not optimal. Compared to previous works that decouple agents in the game by assuming optimality in…
Many real-world domains contain multiple agents behaving strategically with probabilistic transitions and uncertain (potentially infinite) duration. Such settings can be modeled as stochastic games. While algorithms have been developed for…
In this paper, a Nash-type fictitious game framework is introduced to handle a time-inconsistent linear-quadratic optimal control. The Nash-type game in this framework is called fictitious as it is between the decision maker (called real…
This paper proposes a new mathematical paradigm to analyze discrete-time mean-field games. It is shown that finding Nash equilibrium solutions for a general class of discrete-time mean-field games is equivalent to solving an optimization…
We investigate reinforcement learning in the setting of Markov decision processes for a large number of exchangeable agents interacting in a mean field manner. Applications include, for example, the control of a large number of robots…
We present a polynomial-time algorithm that always finds an (approximate) Nash equilibrium for repeated two-player stochastic games. The algorithm exploits the folk theorem to derive a strategy profile that forms an equilibrium by…
Fictitious play (FP) is one of the most fundamental game-theoretical learning frameworks for computing Nash equilibrium in $n$-player games, which builds the foundation for modern multi-agent learning algorithms. Although FP has provable…