Related papers: Distributed Asynchronous Policy Iteration for Sequ…
In this paper, we consider discrete-time infinite horizon problems of optimal control to a terminal set of states. These are the problems that are often taken as the starting point for adaptive dynamic programming. Under very general…
We introduce two algorithms based on a policy iteration method to numerically solve time-dependent Mean Field Game systems of partial differential equations with non-separable Hamiltonians. We prove the convergence of such algorithms in…
We present an accelerated algorithm for the solution of static Hamilton-Jacobi-Bellman equations related to optimal control problems. Our scheme is based on a classic policy iteration procedure, which is known to have superlinear…
A common strategy for solving an unconstrained two-player Nash equilibrium problem with continuous variables is applying Newton's method to the system of nonlinear equations obtained by the corresponding first-order necessary optimality…
In this paper, we propose a new policy iteration algorithm to compute the value function and the optimal controls of continuous time stochastic control problems. The algorithm relies on successive approximations using linear-quadratic…
We consider infinite horizon dynamic programming problems, where the control at each stage consists of several distinct decisions, each one made by one of several agents. In an earlier work we introduced a policy iteration algorithm, where…
We consider a stochastic differential game in the context of forward-backward stochastic differential equations, where one player implements an impulse control while the opponent controls the system continuously. Utilizing the notion of…
Softmax policy gradient is a popular algorithm for policy optimization in single-agent reinforcement learning, particularly since projection is not needed for each gradient update. However, in multi-agent systems, the lack of central…
This work presents a novel policy iteration algorithm to tackle nonzero-sum stochastic impulse games arising naturally in many applications. Despite the obvious impact of solving such problems, there are no suitable numerical methods…
Policy-based methods with function approximation are widely used for solving two-player zero-sum games with large state and/or action spaces. However, it remains elusive how to obtain optimization and statistical guarantees for such…
Continuous-time empirical dynamic discrete choice games offer notable computational advantages over discrete-time models. This paper addresses remaining computational and econometric challenges to further improve both model solution and…
This paper presents a novel method of global adaptive dynamic programming (ADP) for the adaptive optimal control of nonlinear polynomial systems. The strategy consists of relaxing the problem of solving the Hamilton-Jacobi-Bellman (HJB)…
This paper uses value functions to characterize the pure-strategy subgame-perfect equilibria of an arbitrary, possibly infinite-horizon game. It specifies the game's extensive form as a pentaform (Streufert 2023p, arXiv:2107.10801v4), which…
Infinitely repeated games support equilibrium concepts beyond those present in one-shot games (e.g., cooperation in the prisoner's dilemma). Nonetheless, repeated games fail to capture our real-world intuition for settings with many…
Regularization is one of the most important techniques in reinforcement learning algorithms. The well-known soft actor-critic algorithm is a special case of regularized policy iteration where the regularizer is chosen as Shannon entropy.…
This paper discusses algorithms for solving Markov decision processes (MDPs) that have monotone optimal policies. We propose a two-stage alternating convex optimization scheme that can accelerate the search for an optimal policy by…
A recent theory shows that a multi-player decentralized partially observable Markov decision process can be transformed into an equivalent single-player game, enabling the application of \citeauthor{bellman}'s principle of optimality to…
In this work, we study dynamic programming (DP) algorithms for partially observable Markov decision processes with jointly continuous and discrete state-spaces. We consider a class of stochastic systems which have coupled discrete and…
We consider an autonomous navigation problem, whereby a traveler aims at traversing an environment in which an adversary tries to set an ambush. A two players zero sum game is introduced. Players' strategies are computed as random path…
Coordination of distributed agents is required for problems arising in many areas, including multi-robot systems, networking and e-commerce. As a formal framework for such problems, we use the decentralized partially observable Markov…