Related papers: Cascade Markov Decision Processes: Theory and Appl…

Solving optimal control problems to determine a stabilizing controller involves a significant computational effort. Time-varying optimal control provides a remedy by designing a tracking system, given as an ordinary differential equation,…

This note is addressed to giving a short introduction to control theory of stochastic systems, governed by stochastic differential equations in both finite and infinite dimensions. We will mainly explain the new phenomenon and difficulties…

We study an optimal process control problem with multiple assignable causes. The process is initially in-control but is subject to random transition to one of multiple out-of-control states due to assignable causes. The objective is to find…

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterise the value function via HJB equation…

The paper describes a continuous second-variation algorithm to solve optimal control problems where the control is defined on a closed set. A second order expansion of a Lagrangian provides linear updates of the control to construct a…

Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential…

Conditions are established under which the optimal control of processes having both absolutely continuous and singular (with respect to time) controls are equivalent to linear programs over a space of measures on the state and control…

In this paper, we present an optimal control problem for stochastic differential games under Markov regime-switching forward-backward stochastic differential equations with jumps and partial information. First, we prove a sufficient maximum…

This paper is concerned with the maximum principle of stochastic optimal control problems, where the coefficients of the state equation and the cost functional are uncertain, and the system is generally under Markovian regime switching.…

This paper focuses on optimal control problem for a class of discrete-time nonlinear systems. In practical applications, computation time is a crucial consideration when solving nonlinear optimal control problems, especially under real-time…

This paper studies a large number of homogeneous Markov decision processes where the transition probabilities and costs are coupled in the empirical distribution of states (also called mean-field). The state of each process is not known to…

We consider synthesis of control policies that maximize the probability of satisfying given temporal logic specifications in unknown, stochastic environments. We model the interaction between the system and its environment as a Markov…

Our purpose is to study a particular class of optimal stopping problems for Markov processes. We justify the value function convexity and we deduce that there exists a boundary function such that the smallest optimal stopping time is the…

This paper studies continuous-time Markov decision processes under the risk-sensitive average cost criterion. The state space is a finite set, the action space is a Borel space, the cost and transition rates are bounded, and the…

This article considers the average optimality for a continuous-time Markov decision process with Borel state and action spaces and an arbitrarily unbounded nonnegative cost rate. The existence of a deterministic stationary optimal policy is…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

Continuous-time Markov decision processes are an important class of models in a wide range of applications, ranging from cyber-physical systems to synthetic biology. A central problem is how to devise a policy to control the system in order…

The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…

The long-run average payoff per transition (mean payoff) is the main tool for specifying the performance and dependability properties of discrete systems. The problem of constructing a controller (strategy) simultaneously optimizing several…

A study of time homogeneous, real valued Markov processes with a special property and a non-atomic initial distribution is provided. The new notion of a function of evolution of distribution which determines the dependency between one…