Related papers: Ergodic Risk-Sensitive Control of Markov Processes…
The dynamics of the solutions to a class of conservative SPDEs are analysed from two perspectives: Firstly, a probabilistic construction of a corresponding random dynamical system is given for the first time. Secondly, the existence and…
For controlled discrete-time stochastic processes we introduce a new class of dynamic risk measures, which we call process-based. Their main features are that they measure risk of processes that are functions of the history of a base…
We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…
In many applications, the common assumption that a driving noise process affecting a system is independent or Markovian may not be realistic, but the noise process may be assumed to be stationary. To study such problems, this paper…
The aim of this paper is to investigate risk-averse and distributionally robust modeling of Stochastic Optimal Control (SOC) and Markov Decision Process (MDP). We discuss construction of conditional nested risk functionals, a particular…
We observe n possibly dependent random variables, the distribution of which is presumed to be stationary even though this might not be true, and we aim at estimating the stationary distribution. We establish a non-asymptotic deviation bound…
In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constraint that the resulting policy is stabilizing. In practice MDPs are solved based on some form of policy approximation. We will leverage recent…
We study ergodic properties of a class of Markov-modulated general birth-death processes under fast regime switching. The first set of results concerns the ergodic properties of the properly scaled joint Markov process with a parameter that…
We propose a method for approximating solutions to optimization problems involving the global stability properties of parameter-dependent continuous-time autonomous dynamical systems. The method relies on an approximation of the…
We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and…
In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning,…
In the paper adapting Krein Rutman theory we show the existence of solutions to the long run risk sensitive control problem for controlled discrete time Markov processes over locally compact separable metric spaces.
We present some hardness results on finding the optimal policy for the static formulation of distributionally robust Markov decision processes. We construct problem instances such that when the considered policy class is Markovian and…
In this paper the stability and the perturbation bounds of Markov operators acting on abstract state spaces are investigated. Here, an abstract state space is an ordered Banach space where the norm has an additivity property on the cone of…
This paper is concerned with ergodic properties of inhomogeneous Markov processes. Since the transition probabilities depend on initial times, the existing methods to obtain invariant measures for homogeneous Markov processes are not…
The present paper considers the constrained optimal control problem with total undiscounted criteria for a continuous-time Markov decision process (CTMDP) in Borel state and action spaces. Under the standard compactness and continuity…
We propose the entropy of random Markov trajectories originating and terminating at a state as a measure of the stability of a state of a Markov process. These entropies can be computed in terms of the entropy rates and stationary…
The first motivation of this paper is to study stationarity and ergodic properties for a general class of time series models defined conditional on an exogenous covariates process. The dynamic of these models is given by an autoregressive…
This article is concerned with stability and performance of controlled stochastic processes under receding horizon policies. We carry out a systematic study of methods to guarantee stability under receding horizon policies via appropriate…
We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax…