Related papers: Asymptotic Control for a Class of Piecewise Determ…
For controlled discrete-time stochastic processes we introduce a new class of dynamic risk measures, which we call process-based. Their main features are that they measure risk of processes that are functions of the history of a base…
We study stochastic motion planning problems which involve a controlled process, with possibly discontinuous sample paths, visiting certain subsets of the state-space while avoiding others in a sequential fashion. For this purpose, we first…
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision…
We consider the problem of maximizing the expected average reward obtained over an infinite time horizon by $n$ weakly coupled Markov decision processes. Our setup is a substantial generalization of the multi-armed restless bandit problem…
This paper discusses the functional stability of closed-loop Markov Chains under optimal policies resulting from a discounted optimality criterion, forming Markov Decision Processes (MDPs). We investigate the stability of MDPs in the sense…
In this paper, we propose a novel class of Piecewise Deterministic Markov Processes (PDMPs) that are designed to sample from probability distributions $\pi$ supported on a convex set $\mathcal{M}$. This class of PDMPs adapts the concept of…
We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite…
We consider the class of Piecewise Deterministic Markov Processes (PDMP), whose state space is $\R\_{+}^{*}$, that possess an increasing deterministic motion and that shrink deterministically when they jump. Well known examples for this…
In this paper, we investigate the concentration properties of cumulative reward in Markov Decision Processes (MDPs), focusing on both asymptotic and non-asymptotic settings. We introduce a unified approach to characterize reward…
This paper investigates the optimization problem of an infinite stage discrete time Markov decision process (MDP) with a long-run average metric considering both mean and variance of rewards together. Such performance metric is important…
We consider approximate dynamic programming for the infinite-horizon stationary $\gamma$-discounted optimal control problem formalized by Markov Decision Processes. While in the exact case it is known that there always exists an optimal…
Economic Model Predictive Control has recently gained popularity due to its ability to directly optimize a given performance criterion, while enforcing constraint satisfaction for nonlinear systems. Recent research has developed both…
In this paper, we examine applications of the theory of operator-valued processes to algebraic methods in probability theory. We show a central limit theorem for general conservation operator processes. Utilizing this, we analyze the…
We propose to model the records of the maximum Drawdown in capital markets by means a Piecewise Deterministic Markov Process (PDMP). We derive statistical results such as the mean and variance that describes the sequence of maximum Drawdown…
In this article we present a novel discrete-time design approach which reduces the deteriorating effects of sampling on stability and performance in digitally controlled nonlinear mechanical systems. The method is motivated by recent…
We consider finite model approximations of discrete-time partially observed Markov decision processes (POMDPs) under the discounted cost criterion. After converting the original partially observed stochastic control problem to a fully…
A new class of stochastic processes called independent and periodically identically distributed (i.p.i.d.) processes is defined to capture periodically varying statistical behavior. A novel Bayesian theory is developed for detecting a…
A variational formula for the asymptotic variance of general Markov processes is obtained. As application, we get a upper bound of the mean exit time of reversible Markov processes, and some comparison theorems between the reversible and…
This paper studies convergence properties of optimal values and actions for discounted and average-cost Markov Decision Processes (MDPs) with weakly continuous transition probabilities and applies these properties to the stochastic…
This paper is devoted to the analysis of the finite-dimensional distributions and asymptotic behavior of extremal Markov processes connected to the Kendall convolution. In particular, based on its stochastic representation, we provide…