Related papers: Optimal stopping for measure-valued piecewise dete…
Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is often partially…
We study a specific class of finite-horizon mean field optimal stopping problems by means of the dynamic programming approach. In particular, we consider problems where the state process is not affected by the stopping time. Such problems…
We develop methods to solve general optimal stopping problems with opportunities to stop that arrive randomly. Such problems occur naturally in applications with market frictions. Pivotal to our approach is that our methods operate on…
Long-run average optimization problems for Markov decision processes (MDPs) require constructing policies with optimal steady-state behavior, i.e., optimal limit frequency of visits to the states. However, such policies may suffer from…
A general result on the method of randomized stopping is proved. It is applied to optimal stopping of controlled diffusion processes with unbounded coefficients to reduce it to an optimal control problem without stopping. This is motivated…
We investigate a singular-optimal stopping stochastic control problem driven by self-exciting dynamics governed by a Hawkes process. In the continuous-time setting, we show that the optimization problem reduces to solving a variational…
In this paper, we consider a class of stochastic optimal control problems with risk constraints that are expressed as bounded probabilities of failure for particular initial states. We present here a martingale approach that diffuses a risk…
We consider an infinite horizon optimal control problem for a continuous-time Markov chain $X$ in a finite set $I$ with noise-free partial observation. The observation process is defined as $Y_t = h(X_t)$, $t \geq 0$, where $h$ is a given…
In this work, we study dynamic programming (DP) algorithms for partially observable Markov decision processes with jointly continuous and discrete state-spaces. We consider a class of stochastic systems which have coupled discrete and…
The topics treated in this thesis are inherently two-fold. The first part considers the problem of a market maker optimally setting bid/ask quotes over a finite time horizon, to maximize her expected utility. The intensities of the orders…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
This note re-visits the rolling-horizon control approach to the problem of a Markov decision process (MDP) with infinite-horizon discounted expected reward criterion. Distinguished from the classical value-iteration approach, we develop an…
We analyze an optimal stopping problem with a constraint on the expected cost. When the reward function and cost function are Lipschitz continuous in state variable, we show that the value of such an optimal stopping problem is a continuous…
We consider approximate dynamic programming for the infinite-horizon stationary $\gamma$-discounted optimal control problem formalized by Markov Decision Processes. While in the exact case it is known that there always exists an optimal…
This article treats both discrete time and continuous time stopping problems for general Markov processes on the real line with general linear costs. Using an auxiliary function of maximum representation type, conditions are given to…
Stability under model predictive control (MPC) schemes is frequently ensured by terminal ingredients. Employing a (control) Lyapunov function as the terminal cost constitutes a common choice. Learning-based methods may be used to construct…
The objective of this work is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite-time horizon discounted cost. The continuous-time controlled…
We study finite horizon optimal switching problems for hidden Markov chain models under partially observable Poisson processes. The controller possesses a finite range of strategies and attempts to track the state of the unobserved state…
Partially Observable Markov Decision Processes (POMDPs) are systems in which one agent interacts with a stochastic environment, and receives only partial information about the current state. In a multi-environment POMDP (MEPOMDP), the…
We consider the problem of controlling a Markov decision process (MDP) with a large state space, so as to minimize average cost. Since it is intractable to compete with the optimal policy for large scale problems, we pursue the more modest…