Related papers: Solving Zero-Sum Convex Markov Games
In this paper we present a novel generic mapping between Graphical Games and Markov Random Fields so that pure Nash equilibria in the former can be found by statistical inference on the latter. Thus, the problem of deciding whether a…
In this work, we study potential games and Markov potential games under stochastic cost and bandit feedback. We propose a variant of the Frank-Wolfe algorithm with sufficient exploration and recursive gradient estimation, which provably…
This work focuses on the entropy-regularized independent natural policy gradient (NPG) algorithm in multi-agent reinforcement learning. In this work, agents are assumed to have access to an oracle with exact policy evaluation and seek to…
We introduce two min-max problems: the first problem is to minimize the supremum of finitely many rational functions over a compact basic semi-algebraic set whereas the second problem is a 2-player zero-sum polynomial game in randomized…
Policy gradients methods apply to complex, poorly understood, control problems by performing stochastic gradient descent over a parameterized class of polices. Unfortunately, even for simple control problems solvable by standard dynamic…
In noncooperative Nash games, equilibria are often inefficient. This is exemplified by the Prisoner's Dilemma and was first provably shown in the 1980s. Since then, understanding the quality of Nash equilibrium (NE) received considerable…
This work proposes a novel distributed approach for computing a Nash equilibrium in convex games with merely monotone and restricted strongly monotone pseudo-gradients. By leveraging the idea of the centralized operator extrapolation method…
We consider a class of two-player dynamic stochastic nonzero-sum games where the state transition and observation equations are linear, and the primitive random variables are Gaussian. Each controller acquires possibly different dynamic…
Multi-Agent Reinforcement Learning (MARL) -- where multiple agents learn to interact in a shared dynamic environment -- permeates across a wide range of critical applications. While there has been substantial progress on understanding the…
This paper proposes a new mathematical paradigm to analyze discrete-time mean-field games. It is shown that finding Nash equilibrium solutions for a general class of discrete-time mean-field games is equivalent to solving an optimization…
An ideal strategy in zero-sum games should not only grant the player an average reward no less than the value of Nash equilibrium, but also exploit the (adaptive) opponents when they are suboptimal. While most existing works in Markov games…
Nash equilibrium is a central concept in game theory. Several Nash solvers exist, yet none scale to normal-form games with many actions and many players, especially those with payoff tensors too big to be stored in memory. In this work, we…
We investigate a time-inconsistent, non-Markovian finite-player game in continuous time, where each player's objective functional depends non-linearly on the expected value of the state process. As a result, the classical Bellman optimality…
We consider a class of hierarchical noncooperative $N$-player games where the $i$th player solves a parametrized stochastic mathematical program with equilibrium constraints (MPEC) with the caveat that the implicit form of the $i$th…
Adversarial training, a special case of multi-objective optimization, is an increasingly prevalent machine learning technique: some of its most notable applications include GAN-based generative modeling and self-play techniques in…
Zero-sum stochastic games have found important applications in a variety of fields, from machine learning to economics. Work on this model has primarily focused on the computation of Nash equilibrium due to its effectiveness in solving…
Generalized Nash equilibrium (GNE) is a solution concept for complete information games, in which each player's objective function and feasible region depend on other players' actions. While numerical methods for finding GNE when players…
A wide array of modern machine learning applications - from adversarial models to multi-agent reinforcement learning - can be formulated as non-cooperative games whose Nash equilibria represent the system's desired operational states.…
We initiate the study of how to perturb the reward in a zero-sum Markov game with two players to induce a desirable Nash equilibrium, namely arbitrating. Such a problem admits a bi-level optimization formulation. The lower level requires…
Adaptive momentum methods have recently attracted a lot of attention for training of deep neural networks. They use an exponential moving average of past gradients of the objective function to update both search directions and learning…