Related papers: Cascade Markov Decision Processes: Theory and Appl…

This work addresses a switching control problem under which the cost associated with the changes of regimes is allowed to have discontinuities in time. Our main contribution is to show several characterizations of the optimal cost function…

This paper introduces a new approach of treating platoon systems using mean-variance control formulation. The underlying system is a controlled switching diffusion in which the random switching process is a continuous-time Markov chain.…

We are interested in risk constraints for infinite horizon discrete time Markov decision processes (MDPs). Starting with average reward MDPs, we show that increasing concave stochastic dominance constraints on the empirical distribution of…

This paper rigorously connects the problem of optimal control of McKean-Vlasov dynamics with large systems of interacting controlled state processes. Precisely, the empirical distributions of near-optimal control-state pairs for the…

Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and…

We develop a martingale approach for studying continuous-time stochastic differential games of control and stopping, in a non-Markovian framework and with the control affecting only the drift term of the state-process. Under appropriate…

This paper investigates the optimal control problems for the finite-horizon continuous-time Markov decision processes with delay-dependent control policies. We develop compactification methods in decision processes, and show that the…

We establish the existence of an optimal control for a general class of singular control problems with state constraints. The proof uses weak convergence arguments and a time rescaling technique. The existence of optimal controls for…

We introduce a class of models for multidimensional control problems which we call skip-free Markov decision processes on trees. We describe and analyse an algorithm applicable to Markov decision processes of this type that are skip-free in…

In many dynamical systems in nature, the law of the dynamics changes along with the temporal evolution of the system. These changes are often associated with the occurrence of certain events. The timing of occurrence of these events…

We study merchant energy production modeled as a compound switching and timing option. The resulting Markov decision process is intractable. State-of-the-art approximate dynamic programming methods applied to realistic instances of this…

Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…

The purpose of this paper is to provide a detailed probabilistic analysis of the optimal control of nonlinear stochastic dynamical systems of the McKean Vlasov type. Motivated by the recent interest in mean field games, we highlight the…

For Markov processes over discrete configurations, an asymptotic bound on the uncertainty of stochastic fluxes is derived in terms of the harmonic mean of decay rates with respect to the stationary distribution. This bound is necessarily…

A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…

We examine a constrained Markov decision process under uncertain transition probabilities, with the uncertainty modeled as deviations from observed transition probabilities. We construct the uncertainty set associated with the deviations…

Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…

The study of density-dependent stochastic population processes is important from a historical perspective as well as from the perspective of a number of existing and emerging applications today. In more recent applications of these…

We present a method to sample Markov-chain trajectories constrained to both the initial and final conditions, which we term Markov bridges. The trajectories are conditioned to end in a specific state at a given time. We derive the master…

We give a short overview of recent results on a specific class of Markov process: the Piecewise Deterministic Markov Processes (PDMPs). We first recall the definition of these processes and give some general results. On more specific cases…