Related papers: Asymptotic Analysis and Practical Evaluation of Ju…
We present a short introduction into the framework of piecewise deterministic Markov processes. We illustrate the abstract mathematical setting with a series of examples related to dispersal of biological systems, cell cycle models, gene…
This paper extends to Continuous-Time Jump Markov Decision Processes (CTJMDP) the classic result for Markov Decision Processes stating that, for a given initial state distribution, for every policy there is a (randomized) Markov policy,…
We prove a large-deviation principle (LDP) for the sample paths of jump Markov processes in the small noise limit when, possibly, all the jump rates vanish uniformly, but slowly enough, in a region of the state space. We further discuss the…
The standard approach to analyzing the asymptotic complexity of probabilistic programs is based on studying the asymptotic growth of certain expected values (such as the expected termination time) for increasing input size. We argue that…
Inspired by a duration-dependent life insurance model, we consider continuous-time semi-Markov jump processes, initially assumed to have a finite state-space. We develop approximations using jump processes that are time-homogeneous Markov,…
We consider parametric Markov decision processes (pMDPs) that are augmented with unknown probability distributions over parameter values. The problem is to compute the probability to satisfy a temporal logic specification with any concrete…
The paper deals with the asymptotic properties of a random jump process in a high contrast periodic medium in $\mathbb R^d$, $d\geq 1$. We show that if the coordinates of the random jump process in $\mathbb R^d$ are equipped with an extra…
We consider a unified framework of sequential change-point detection and hypothesis testing modeled by means of hidden Markov chains. One observes a sequence of random variables whose distributions are functionals of a hidden Markov chain.…
Asymptotic theory for approximate martingale estimating functions is generalised to diffusions with finite-activity jumps, when the sampling frequency and terminal sampling time go to infinity. Rate optimality and efficiency are of…
This paper investigates asymptotic behaviors of gradient descent algorithms (particularly accelerated gradient descent and stochastic gradient descent) in the context of stochastic optimization arising in statistics and machine learning…
We extend Monte Carlo samplers based on piecewise deterministic Markov processes (PDMP samplers) by formally defining different boundary conditions such as sticky floors, soft and hard walls and teleportation portals. This allows PDMP…
Diffusive approximations of Markov jump processes often fail to accurately capture large fluctuations. This is confounding, as the rare events triggered by these large fluctuations, such as the failure of electronic memories, are often the…
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In…
This paper investigates the optimization problem of an infinite stage discrete time Markov decision process (MDP) with a long-run average metric considering both mean and variance of rewards together. Such performance metric is important…
We treat the class of universal Markov processes on the d-dimensional Euklidean space which do not depend on random. For these, as well as for several subclasses, we prove criteria whether a function f, defined on the positive half-line,…
In [20], the authors addressed the question of the averaging of a slow-fast Piecewise Deterministic Markov Process (PDMP) in infinite dimension. In the present paper, we carry on and complete this work by the mathematical analysis of the…
Partially observable Markov decision processes (POMDPs) provide an elegant mathematical framework for modeling complex decision and planning problems in stochastic domains in which states of the system are observable only indirectly, via a…
We aim at characterizing viability, invariance and some reachability properties of controlled piecewise deterministic Markov processes (PDMPs). Using analytical methods from the theory of viscosity solutions, we establish criteria for…
We estimate a general mixture of Markov jump processes. The key novel feature of the proposed mixture is that the transition intensity matrices of the Markov processes comprising the mixture are entirely unconstrained. The Markov processes…
This article is concerned with stability analysis and stabilization of randomly switched nonlinear systems. These systems may be regarded as piecewise deterministic stochastic systems: the discrete switches are triggered by a stochastic…