Related papers: A policy iteration method for Mean Field Games
We present an accelerated algorithm for the solution of static Hamilton-Jacobi-Bellman equations related to optimal control problems. Our scheme is based on a classic policy iteration procedure, which is known to have superlinear…
Mean-field games arise in various fields including economics, engineering, and machine learning. They study strategic decision making in large populations where the individuals interact via certain mean-field quantities. The ground metrics…
Mean field games and controls involve guiding the behavior of large populations of interacting agents, where each individual's influence on the group is negligible but collectively impacts overall dynamics. Hybrid systems integrate…
We propose a reinforcement learning algorithm for stationary mean-field games, where the goal is to learn a pair of mean-field state and stationary policy that constitutes the Nash equilibrium. When viewing the mean-field state and the…
We propose a single-level numerical approach to solve Stackelberg mean field game (MFG) problems. In Stackelberg MFG, an infinite population of agents play a non-cooperative game and choose their controls to optimize their individual…
We establish the convergence of the deep actor-critic reinforcement learning algorithm presented in [Angiuli et al., 2023a] in the setting of continuous state and action spaces with an infinite discrete-time horizon. This algorithm provides…
Multi-agent reinforcement learning, despite its popularity and empirical success, faces significant scalability challenges in large-population dynamic games. Graphon mean field games (GMFGs) offer a principled framework for approximating…
The aim of this paper is to study first order Mean field games subject to a linear controlled dynamics on $\mathbb R^{d}$. For this kind of problems, we define Nash equilibria (called Mean Field Games equilibria), as Borel probability…
We consider mean field games with discrete state spaces (called discrete mean field games in the following) and we analyze these games in continuous and discrete time, over finite as well as infinite time horizons. We prove the existence of…
We develop the fictitious play algorithm in the context of the linear programming approach for mean field games of optimal stopping and mean field games with regular control and absorption. This algorithm allows to approximate the mean…
While the topic of mean-field games (MFGs) has a relatively long history, heretofore there has been limited work concerning algorithms for the computation of equilibrium control policies. In this paper, we develop a computable policy…
Entropy regularization has been extensively adopted to improve the efficiency, the stability, and the convergence of algorithms in reinforcement learning. This paper analyzes both quantitatively and qualitatively the impact of entropy…
We investigate reinforcement learning in the setting of Markov decision processes for a large number of exchangeable agents interacting in a mean field manner. Applications include, for example, the control of a large number of robots…
We study convergence rates of the generalized conditional gradient (GCG) method applied to fully discretized Mean Field Games (MFG) systems. While explicit convergence rates of the GCG method have been established at the continuous PDE…
Mean Field Game systems describe equilibrium configurations in differential games with infinitely many infinitesimal interacting agents. We introduce a learning procedure (similar to the Fictitious Play) for these games and show its…
Mean Field Games (MFG) have been introduced to tackle games with a large number of competing players. Considering the limit when the number of players is infinite, Nash equilibria are studied by considering the interaction of a typical…
Reinforcement learning is a powerful tool to learn the optimal policy of possibly multiple agents by interacting with the environment. As the number of agents grow to be very large, the system can be approximated by a mean-field problem.…
In this article, two methods for solving mean-field type optimal control problems are proposed and investigated. The two methods are iterative methods: at each iteration, a Hamilton-Jacobi-Bellman equation is solved, for a terminal…
This paper is concerned with the convergence rate of policy iteration for (deterministic) optimal control problems in continuous time. To overcome the problem of ill-posedness due to lack of regularity, we consider a semi-discrete scheme by…
Policy-based methods with function approximation are widely used for solving two-player zero-sum games with large state and/or action spaces. However, it remains elusive how to obtain optimization and statistical guarantees for such…