Related papers: On gradual-impulse control of continuous-time Mark…
Planning for distributed agents with partial state information is considered from a decision- theoretic perspective. We describe generalizations of both the MDP and POMDP models that allow for decentralized control. For even a small number…
This paper, based on the compactness-continuity and finite value conditions, establishes the sufficiency of the class of stationary policies out of the general class of history-dependent ones for a constrained continuous-time Markov…
A mean-field selective optimal control problem of multipopulation dynamics via transient leadership is considered. The agents in the system are described by their spatial position and their probability of belonging to a certain population.…
We consider a dynamic programming (DP) approach to approximately solving an infinite-horizon constrained Markov decision process (CMDP) problem with a fixed initial-state for the expected total discounted-reward criterion with a…
We study an optimal process control problem with multiple assignable causes. The process is initially in-control but is subject to random transition to one of multiple out-of-control states due to assignable causes. The objective is to find…
In this paper, we study asymptotic properties of problems of control of stochastic discrete time systems (also known as Markov decision processes) with time averaging and time discounting optimality criteria, and we establish that the…
We consider a discrete-time Markov decision process with Borel state and action spaces. The performance criterion is to maximize a total expected {utility determined by unbounded return function. It is shown the existence of optimal…
The main goal of this paper is to derive sufficient conditions for the existence of an optimal control strategy for the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a…
We consider a stochastic impulse control problem that is motivated by applications such as the optimal exploitation of a natural resource. In particular, we consider a stochastic system whose uncontrolled state dynamics are modelled by a…
In many operations management problems, we need to make decisions sequentially to minimize the cost while satisfying certain constraints. One modeling approach to study such problems is constrained Markov decision process (CMDP). When…
We present existence and discrete-time approximation results on optimal control policies for continuous-time stochastic control problems under a variety of information structures. These include fully observed models, partially observed…
This paper studies constrained optimal impulse control problems of a deterministic system described by a (semi)flow, where the performance measures are the discounted total costs including both the costs incurred with applying impulses as…
We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…
The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term 'risk-sensitive' refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk.…
This paper studies the optimization of Markov decision processes (MDPs) from a risk-seeking perspective, where the risk is measured by conditional value-at-risk (CVaR). The objective is to find a policy that maximizes the long-run CVaR of…
We consider the synthesis of control policies for probabilistic systems, modeled by Markov decision processes, operating in partially known environments with temporal logic specifications. The environment is modeled by a set of Markov…
In this paper, we show how a simulated Markov decision process (MDP) built by the so-called \emph{baseline} policies, can be used to compute a different policy, namely the \emph{simulated optimal} policy, for which the performance of this…
This note describes sufficient conditions under which total-cost and average-cost Markov decision processes (MDPs) with general state and action spaces, and with weakly continuous transition probabilities, can be reduced to discounted MDPs.…
We provide an overview on how to use the measurable selection techniques to derive the dynamic programming principle for a general stochastic optimal control/stopping problem. By considering its martingale problem formulation on the…
In this paper, we consider a discrete-time Markov Decision Process (MDP) on a finite state-action space with a long-run risk-sensitive criterion used as the objective function. We discuss the concept of Blackwell optimality and comment on…