Related papers: Ergodic Risk-Sensitive Control of Markov Processes…
We investigated a cost-constrained static ergodic control problem of the variance of measure-valued affine processes and its application in streamflow management. The controlled system is a jump-driven mixed moving average process that…
We explore the concept of a consistent exchangeable survival process - a joint distribution of survival times in which the risk set evolves as a continuous-time Markov process with homogeneous transition rates. We show a correspondence with…
The class of nonlinear Markov processes is characterized by the dependence of the current state of the process on its current distribution in addition to the dependence on the previous state. Due to this feature, these processes are…
We study discrete-time discounted constrained Markov decision processes (CMDPs) on Borel spaces with unbounded reward functions. In our approach the transition probability functions are weakly or set-wise continuous. The reward functions…
It is known that Dobrushin's ergodicity coefficient is one of the effective tools in the investigations of limiting behavior of Markov processes. Several interesting properties of the ergodicity coefficient of a positive mapping defined on…
We present stability conditions for deterministic time-varying nonlinear discrete-time systems whose inputs aim to minimize an infinite-horizon time-dependent cost. Global asymptotic and exponential stability properties for general…
It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…
We provide a probabilistic analysis of the banker algorithm when transition probabilities may depend on time and space. The transition probabilities evolve, as time goes by, along the trajectory of an ergodic Markovian environment, whereas…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
In this review/tutorial article, we present recent progress on optimal control of partially observed Markov Decision Processes (POMDPs). We first present regularity and continuity conditions for POMDPs and their belief-MDP reductions, where…
We consider minimizing the probability of falling below a target growth rate of the wealth process up to a time horizon $T$ in an incomplete market model, and then study the asymptotic behavior of minimizing probability as $T\to\infty$.…
This paper considers the problem of stabilizing a discrete-time non-linear stochastic system over a finite capacity noiseless channel. Our focus is on systems which decompose into a stable and unstable component, and the stability notion…
A discrete-time method for solving problems in optimal quantum control is presented. Controlling the time discretized markovian dynamics of a quantum system can be reduced to a Markov-decision process. We demonstrate this method in this…
In this paper, we focus on formal synthesis of control policies for finite Markov decision processes with non-negative real-valued costs. We develop an algorithm to automatically generate a policy that guarantees the satisfaction of a…
We develop a method for computing policies in Markov decision processes with risk-sensitive measures subject to temporal logic constraints. Specifically, we use a particular risk-sensitive measure from cumulative prospect theory, which has…
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision…
The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…
We consider stationary stochastic dynamical systems evolving on a compact metric space, by perturbing a deterministic dynamics with a random noise, added according to an arbitrary probabilistic distribution. We prove the maximal and…
A dynamical model consists of a continuous self-map $T: \mathcal{X} \to \mathcal{X}$ of a compact state space $\mathcal{X}$ and a continuous observation function $f: \mathcal{X} \to \mathbb{R}$. This paper considers the fitting of a…
We prove that the class of discrete time stationary max-stable process satisfying the Markov property is equal, up to time reversal, to the class of stationary max-autoregressive processes of order $1$. A similar statement is also proved…