Related papers: Mean Field Markov Decision Processes
Mean field optimal control problems are a class of optimization problems that arise from optimal control when applied to the many body setting. In the noisy case one has a set of controllable stochastic processes and a cost function that is…
We study the convergence of Markov Decision Processes made of a large number of objects to optimization problems on ordinary differential equations (ODE). We show that the optimal reward of such a Markov Decision Process, satisfying a…
This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises…
We introduce a mean field game with rank-based reward: competing agents optimize their effort to achieve a goal, are ranked according to their completion time, and paid a reward based on their relative rank. First, we propose a tractable…
In this work, we systematically investigate mean field games and mean field type control problems with multiple populations using a coupled system of forward-backward stochastic differential equations of McKean-Vlasov type stemming from…
We study a multi-agent mean field type control problem in discrete time where the agents aim to find a socially optimal strategy and where the state and action spaces for the agents are assumed to be continuous. The agents are only weakly…
This paper focuses on the role of a government of a large population of interacting agents as a mean field optimal control problem derived from deterministic finite agent dynamics. The control problems are constrained by a PDE of…
We consider a class of optimal control problems that arise in connection with optimal advertising under uncertainty. Two main features appear in the model: a delay in the control variable driving the state dynamics; a mean-field term both…
This paper analyzes and explicitly solves a class of long-term average impulse control problems with a specific mean-field interaction. The underlying process is a general one-dimensional diffusion with appropriate boundary behavior. The…
We derive a framework to compute optimal controls for problems with states in the space of probability measures. Since many optimal control problems constrained by a system of ordinary differential equations (ODE) modelling interacting…
We study infinite horizon discounted Mean Field Control (MFC) problems with common noise through the lens of Mean Field Markov Decision Processes (MFMDP). We allow the agents to use actions that are randomized not only at the individual…
We are interested in risk constraints for infinite horizon discrete time Markov decision processes (MDPs). Starting with average reward MDPs, we show that increasing concave stochastic dominance constraints on the empirical distribution of…
We present the first finite time global convergence analysis of policy gradient in the context of infinite horizon average reward Markov decision processes (MDPs). Specifically, we focus on ergodic tabular MDPs with finite state and action…
We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we…
We consider the problem of maximizing the expected average reward obtained over an infinite time horizon by $n$ weakly coupled Markov decision processes. Our setup is a substantial generalization of the multi-armed restless bandit problem…
The most relevant problems in discounted reinforcement learning involve estimating the mean of a function under the stationary distribution of a Markov reward process, such as the expected return in policy evaluation, or the policy gradient…
Recent work linking deep neural networks and dynamical systems opened up new avenues to analyze deep learning. In particular, it is observed that new insights can be obtained by recasting deep learning as an optimal control problem on…
We study a specific class of finite-horizon mean field optimal stopping problems by means of the dynamic programming approach. In particular, we consider problems where the state process is not affected by the stopping time. Such problems…
Mean field control (MFC) problems have been introduced to study social optima in very large populations of strategic agents. The main idea is to consider an infinite population and to simplify the analysis by using a mean field…
We study a family of McKean-Vlasov (mean-field) type ergodic optimal control problems with linear control, and quadratic dependence on control of the cost function. For this class of problems we establish existence and uniqueness of an…