Related papers: Entropy Regularization for Mean Field Games with L…
We present the development and analysis of a reinforcement learning (RL) algorithm designed to solve continuous-space mean field game (MFG) and mean field control (MFC) problems in a unified manner. The proposed approach pairs the…
Estimating the unknown reward functions driving agents' behaviors is of central interest in inverse reinforcement learning and game theory. To tackle this problem, we develop a unified framework for reward function recovery in two-player…
The policy iteration method is a classical algorithm for solving optimal control problems. In this paper, we introduce a policy iteration method for Mean Field Games systems, and we study the convergence of this procedure to a solution of…
In this work, we study the contraction conditions of iterative algorithms for stationary and finite-horizon discrete-time regularized mean-field games (MFGs) with multiple populations, where each population only interacts with the state…
A novel Policy Gradient (PG) algorithm, called $\textit{Matryoshka Policy Gradient}$ (MPG), is introduced and studied, in the context of fixed-horizon max-entropy reinforcement learning, where an agent aims at maximizing entropy bonuses…
Training LLM agents in multi-turn environments with sparse rewards, where completing a single task requires 30+ turns of interaction within an episode, presents a fundamental challenge for reinforcement learning. We identify a critical…
In this paper we study iterative procedures for stationary equilibria in games with large number of players. Most of learning algorithms for games with continuous action spaces are limited to strict contraction best reply maps in which the…
Two-player games such as board games have long been used as traditional benchmarks for reinforcement learning. This work revisits a policy optimization method with reverse Kullback-Leibler regularization and entropy regularization and…
For RL algorithms, appropriate entropy control is crucial to their effectiveness. To control the policy entropy, a commonly used method is entropy regularization, which is adopted in various popular RL algorithms including PPO, SAC and A3C.…
The well-posedness of a multi-population dynamical system with an entropy regularization and its convergence to a suitable mean-field approximation are proved, under a general set of assumptions. Under further assumptions on the evolution…
We study infinite horizon discounted Mean Field Control (MFC) problems with common noise through the lens of Mean Field Markov Decision Processes (MFMDP). We allow the agents to use actions that are randomized not only at the individual…
Stochastic games provide a framework for interactions among multiple agents and enable a myriad of applications. In these games, agents decide on actions simultaneously, the state of every agent moves to the next state, and each agent…
Regularization of control policies using entropy can be instrumental in adjusting predictability of real-world systems. Applications benefiting from such approaches range from, e.g., cybersecurity, which aims at maximal unpredictability, to…
In the Lasry--Lions framework, Mean-Field Games (MFGs) model interactions among an infinite number of agents. However, existing algorithms either require strict monotonicity or only guarantee the convergence of averaged iterates, as in…
We address in this paper a fundamental question that arises in mean-field games (MFGs), namely whether mean-field equilibria (MFE) for discrete-time finite-horizon MFGs can be used to obtain approximate stationary as well as non-stationary…
A mean-field game (MFG) seeks the Nash Equilibrium of a game involving a continuum of players, where the Nash Equilibrium corresponds to a fixed point of the best-response mapping. However, simple fixed-point iterations do not always…
This paper studies the convergence of mean field games with finite state space to mean field games with a continuous state space. We examine a space discretization of a diffusive dynamics, which is reminiscent of the Markov chain…
We propose a single-level numerical approach to solve Stackelberg mean field game (MFG) problems. In Stackelberg MFG, an infinite population of agents play a non-cooperative game and choose their controls to optimize their individual…
Mean-field games (MFG) have become significant tools for solving large-scale multi-agent reinforcement learning problems under symmetry. However, the assumption of exact symmetry limits the applicability of MFGs, as real-world scenarios…
We consider a class of mean field games in which the agents interact through both their states and controls, and we focus on situations in which a generic agent tries to adjust her speed (control) to an average speed (the average is made in…