Related papers: Time-inconsistent mean-field stopping problems: A …
Stochastic and soft optimal policies resulting from entropy-regularized Markov decision processes (ER-MDP) are desirable for exploration and imitation learning applications. Motivated by the fact that such policies are sensitive with…
This work is devoted to finding the closed-loop equilibria for a class of mean-field games (MFGs) with infinitely many symmetric players in a common switching environment when the cost functional is under general discount in time. There are…
Time change is a powerful technique for generating noises and providing flexible models. In the framework of time changed Brownian and Poisson random measures we study the existence and uniqueness of a solution to a general mean-field…
In this paper, we consider discrete-time dynamic games of the mean-field type with a finite number $N$ of agents subject to an infinite-horizon discounted-cost optimality criterion. The state space of each agent is a locally compact Polish…
Canonical models of Markov decision processes (MDPs) usually consider geometric discounting based on a constant discount factor. While this standard modeling approach has led to many elegant results, some recent studies indicate the…
Long-run average optimization problems for Markov decision processes (MDPs) require constructing policies with optimal steady-state behavior, i.e., optimal limit frequency of visits to the states. However, such policies may suffer from…
This paper revisits the well-studied \emph{optimal stopping} problem but within the \emph{large-population} framework. In particular, two classes of optimal stopping problems are formulated by taking into account the \emph{relative…
This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems,…
Discount regularization, using a shorter planning horizon when calculating the optimal policy, is a popular choice to restrict planning to a less complex set of policies when estimating an MDP from sparse or noisy data (Jiang et al., 2015).…
We consider a dynamic programming (DP) approach to approximately solving an infinite-horizon constrained Markov decision process (CMDP) problem with a fixed initial-state for the expected total discounted-reward criterion with a…
Entropy regularization has been extensively adopted to improve the efficiency, the stability, and the convergence of algorithms in reinforcement learning. This paper analyzes both quantitatively and qualitatively the impact of entropy…
Mean field games (MFGs) offer a powerful framework for modeling large-scale multi-agent systems. This paper addresses MFGs formulated in continuous time with discrete state spaces, where agents' dynamics are governed by continuous-time…
Congestion pricing has become an effective instrument for traffic demand management on road networks. This paper proposes an optimal control approach for congestion pricing for day-to-day timescale that incorporates demand uncertainty and…
In optimal stopping problems, a Markov structure guarantees Markovian optimal stopping times (first exit times). Surprisingly, there is no analogous result for Markovian stopping games once randomization is required. This paper addresses…
This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be…
In a probabilistic mean field game driven by a L\'evy process an individual player aims to minimize a long run discounted/ergodic cost by controlling the process through a pair of increasing and decreasing c\`adl\`ag processes, while he is…
We consider three equilibrium concepts proposed in the literature for time-inconsistent stopping problems, including mild equilibria, weak equilibria and strong equilibria. The discount function is assumed to be log sub-additive and the…
This paper studies the convergence of mean field games with finite state space to mean field games with a continuous state space. We examine a space discretization of a diffusive dynamics, which is reminiscent of the Markov chain…
We study a general formulation of the classical two-player Dynkin game in a discrete time Markovian setting. We identify an appropriate class of mixed strategies -- \textit{Markovian randomized stopping times} -- in which players stop at…
We consider the problem of constrained Markov Decision Process (CMDP) where an agent interacts with a unichain Markov Decision Process. At every interaction, the agent obtains a reward. Further, there are $K$ cost functions. The agent aims…