Related papers: Impulsive control for continuous-time Markov Decis…
We consider a discrete-time Markov decision process with Borel state and action spaces. The performance criterion is to maximize a total expected {utility determined by unbounded return function. It is shown the existence of optimal…
This paper is concerned with stochastic impulse control problems in which the running cost changes depending on the impulse control. Because of such a dependence, it brings several difficulties when the usual dynamic programming principle…
Navigating a collision-free and optimal trajectory for a robot is a challenging task, particularly in environments with moving obstacles such as humans. We formulate this problem as a stochastic optimal control problem. Since solving the…
We analyze the infinite horizon minimax average cost Markov Control Model (MCM), for a class of controlled process conditional distributions, which belong to a ball, with respect to total variation distance metric, centered at a known…
In this paper, we establish a general stochastic maximum principle for optimal control for systems described by a continuous-time Markov regime-switching stochastic recursive utilities model. The control domain is postulated not to be…
We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is…
In this paper we consider a broad class of infinite horizon discrete-time optimal control models that involve a nonnegative cost function and an affine mapping in their dynamic programming equation. They include as special cases classical…
We address the attack impact evaluation problem for control system security. We formulate the problem as a Markov decision process with a temporally joint chance constraint that forces the adversary to avoid being detected throughout the…
We consider both discrete and continuous "uncertain horizon" deterministic control processes, for which the termination time is a random variable. We examine the dynamic programming equations for the value function of such processes,…
We study time-inconsistent recursive stochastic control problems, i.e., for which the Bellman principle of optimality does not hold. For this class of problems classical optimal controls may fail to exist, or to be relevant in practice, and…
Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is often partially…
We consider an infinite horizon optimal control problem for a continuous-time Markov chain $X$ in a finite set $I$ with noise-free partial observation. The observation process is defined as $Y_t = h(X_t)$, $t \geq 0$, where $h$ is a given…
In this paper, a general stochastic model with controls applied at the moments when the random process hits the boundary of a given subset of the state set is proposed and studied. The general concept of the model is formulated and its…
Calculating optimal policies is known to be computationally difficult for Markov decision processes (MDPs) with Borel state and action spaces. This paper studies finite-state approximations of discrete time Markov decision processes with…
We consider an infinite horizon discounted optimal control problem for piecewise deterministic Markov processes, where a piecewise open-loop control acts continuously on the jump dynamics and on the deterministic flow. For this class of…
This paper is devoted to studying an infinite time horizon stochastic recursive control problem with jumps, where infinite time horizon stochastic differential equation and backward stochastic differential equation with jumps describe the…
In this paper, we propose a new policy iteration algorithm to compute the value function and the optimal controls of continuous time stochastic control problems. The algorithm relies on successive approximations using linear-quadratic…
We present an elementary state augmentation method for a class of static risk measure applied to the total cost for both Markov decision processes and stochastic optimal control, such that dynamic programming equations can be derived on the…
Interval Markov decision processes are a class of Markov models where the transition probabilities between the states belong to intervals. In this paper, we study the problem of efficient estimation of the optimal policies in Interval…
This paper concentrates on the minimal hitting probability of continuous-time controlled Markov systems (CTCMSs) with countable state and finite admissible action spaces. The existence of an optimal policy is first proved. In particular,…