Related papers: Cascade Markov Decision Processes: Theory and Appl…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
The paper proposes a new stochastic intervention control model conducted in various commodity and stock markets. The essence of the phenomenon of intervention is described in accordance with current economic theory. A review of papers on…
In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to…
In this paper we present a dynamic programing approach to stochastic optimal control problems with dynamic, time-consistent risk constraints. Constrained stochastic optimal control problems, which naturally arise when one has to consider…
This paper focuses on a class of continuous-time controlled Markov chains with time-inconsistent and distribution-dependent cost functional (in some appropriate sense). A new definition of time-inconsistent distribution-dependent…
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…
Trajectory optimization is a fundamental stochastic optimal control problem. This paper deals with a trajectory optimization approach for dynamical systems subject to measurement noise that can be fitted into linear time-varying stochastic…
We investigate a singular-optimal stopping stochastic control problem driven by self-exciting dynamics governed by a Hawkes process. In the continuous-time setting, we show that the optimization problem reduces to solving a variational…
This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems,…
We use classical tools from calculus of variations to formally derive necessary conditions for a Markov control to be optimal in a standard finite time horizon stochastic control problem. As an example, we solve the well-known Merton…
This paper concerns some time optimal control problems of three different ordinary differential equations in $\mathbb{R}^2$. Corresponding to certain initial data and controls, the solutions of the systems quench at finite time. The goal to…
We have shown recently that a Markov process conditioned on rare events involving time-integrated random variables can be described in the long-time limit by an effective Markov process, called the driven process, which is given…
Consider the following multi-phase project management problem. Each project is divided into several phases. All projects enter the next phase at the same point chosen by the decision maker based on observations up to that point. Within each…
We investigate discrete-time mean-variance portfolio selection problems viewed as a Markov decision process. We transform the problems into a new model with deterministic transition function for which the Bellman optimality equation holds.…
The objective of this work is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite-time horizon discounted cost. The continuous-time controlled…
Many decision problems in economics, information technology, and industry can be transformed to an optimal stopping of adapted random vectors with some utility function over the set of Markov times with respect to filtration build by the…
We consider the control of a Markov decision process (MDP) that undergoes an abrupt change in its transition kernel (mode). We formulate the problem of minimizing regret under control-switching based on mode change detection, compared to a…
Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend…