Related papers: A Policy Iteration Algorithm for N-player General-…
We construct a statistical ensemble of games, where in each independent subensemble we have two players playing the same game. We derive the mean payoffs per move of the representative players of the game, and we evaluate all the…
We address the numerical solution of second-order Mean Field Game problems through Newton iterations in infinite dimensions, introduced in [14], where quadratic convergence of the method was rigorously established. Building upon this…
The goal of this paper is to investigate new and simple convergence analysis of dynamic programming for linear quadratic regulator problem of discrete-time linear time-invariant systems. In particular, bounds on errors are given in terms of…
The convergence of online learning algorithms in games under self-play is a fundamental question in game theory and machine learning. Among various notions of convergence, last-iterate convergence is particularly desirable, as it reflects…
In this paper, we consider a learning problem among non-cooperative agents interacting in a time-varying system. Specifically, we focus on repeated linear quadratic network games, in which the network of interactions changes with time and…
This paper proposes a new method for finding closed-loop saddle points in zero-sum linear-quadratic stochastic differential games by decoupling their inherent structure. Specifically, we develop a nested iterative scheme that constructs a…
Markov Decision Processes are classically solved using Value Iteration and Policy Iteration algorithms. Recent interest in Reinforcement Learning has motivated the study of methods inspired by optimization, such as gradient ascent. Among…
Simple stochastic games are turn-based 2.5-player zero-sum graph games with a reachability objective. The problem is to compute the winning probability as well as the optimal strategies of both players. In this paper, we compare the three…
Self-serving, rational agents sometimes cooperate to their mutual benefit. The two-player iterated prisoner's dilemma game is a model for including the emergence of cooperation. It is generally believed that there is no simple ultimatum…
This work proposes a policy learning algorithm for seeking generalised feedback Nash equilibria (GFNE) in $N_P$-player noncooperative dynamic games. We consider linear-quadratic games with stochastic dynamics and design a best-response…
This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters. We show that the approximate…
We study Stackelberg equilibria in finitely repeated games, where the leader commits to a strategy that picks actions in each round and can be adaptive to the history of play (i.e. they commit to an algorithm). In particular, we study…
Zero-sum Linear Quadratic (LQ) games are fundamental in optimal control and can be used (i)~as a dynamic game formulation for risk-sensitive or robust control and (ii)~as a benchmark setting for multi-agent reinforcement learning with two…
Strategy iteration is a technique frequently used for two-player games in order to determine the winner or compute payoffs, but to the best of our knowledge no general framework for strategy iteration has been considered. Inspired by…
We propose a method for designing policies for convex stochastic control problems characterized by random linear dynamics and convex stage cost. We consider policies that employ quadratic approximate value functions as a substitute for the…
We propose policy gradient algorithms for robust infinite-horizon Markov decision processes (MDPs) with non-rectangular uncertainty sets, thereby addressing an open challenge in the robust MDP literature. Indeed, uncertainty sets that…
We study two-player general sum repeated finite games where the rewards of each player are generated from an unknown distribution. Our aim is to find the egalitarian bargaining solution (EBS) for the repeated game, which can lead to much…
We consider inexact policy iteration methods for large-scale infinite-horizon discounted MDPs with finite spaces, a variant of policy iteration where the policy evaluation step is implemented inexactly using an iterative solver for linear…
In this paper, we consider an infinite horizon average reward Markov Decision Process (MDP). Distinguishing itself from existing works within this context, our approach harnesses the power of the general policy gradient-based algorithm,…
In this paper, we study finite-agent linear-quadratic games on graphs. Specifically, we propose a comprehensive framework that extends the existing literature by incorporating heterogeneous and interpretable player interactions. Compared to…