Related papers: Pitman's discrete $2M-X$ theorem for arbitrary ini…
We consider a change-point detection problem for a simple class of Piecewise Deterministic Markov Processes (PDMPs). A continuous-time PDMP is observed in discrete time and through noise, and the aim is to propose a numerical method to…
We prove a stochastic averaging theorem for stochastic differential equations in which the slow and the fast variables interact. The approximate Markov fast motion is a family of Markov process with generator ${\mathcal L}_x$ for which we…
Based on discrete observations $X_0,X_{\Delta},\dots, X_{n\Delta}$ for $\Delta=n^{-\gamma}$ with $\gamma\in [0,1)$ of the null-recurrent dynamic $dX_t = \sigma(X_t)dW_t$ with a Brownian motion $W$ and $\sigma(x)=\alpha\mathbb{1}\{x<\rho\} +…
In this paper, we are concerned with centered Markov Additive Processes $\{(X_t,Y_t)\}_{t\in\T}$ where the driving Markov process $\{X_t\}_{t\in\T}$ has a finite state space. Under suitable conditions, we provide a local limit theorem for…
The Pitman-Yor process is a random discrete measure. The random weights or masses follow the two-parameter Poisson-Dirichlet distribution with parameters $0<\alpha<1, \theta>-\alpha$. The parameters $\alpha$ and $\theta$ correspond to the…
Discrete time stochastic optimal control problems and Markov decision processes (MDPs), respectively, serve as fundamental models for problems that involve sequential decision making under uncertainty and as such constitute the theoretical…
In this work, we generalize the concept of bisimulation metric in order to metrize the behaviour of continuous-time processes. Similarly to what is done for discrete-time systems, we follow two approaches and show that they coincide: as a…
This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…
In this work we consider time series with a finite number of discrete point changes. We assume that the data in each segment follows a different probability density functions (pdf). We focus on the case where the data in all segments are…
We develop a pseudo-metric analogue of bisimulation for generalized semi-Markov processes. The kernel of this pseudo-metric corresponds to bisimulation; thus we have extended bisimulation for continuous-time probabilistic processes to a…
We present limit theorems for a sequence of Piecewise Deterministic Markov Processes (PDMPs) taking values in a separable Hilbert space. This class of processes provides a rigorous framework for stochastic spatial models in which discrete…
We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location---inter-arrival time Markov…
In [20], the authors addressed the question of the averaging of a slow-fast Piecewise Deterministic Markov Process (PDMP) in infinite dimension. In the present paper, we carry on and complete this work by the mathematical analysis of the…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
In this paper we study coupled fully non-local equations, where a linear non-local operator jointly acts on the time and space variables. We establish existence and uniqueness of the solution. A maximum principle is proved and used to…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this paper, we propose a novel solution to this decision-theoretic planning problem that directly…
This paper studies function approximation for finite horizon discrete time Markov decision processes under certain convexity assumptions. Uniform convergence of these approximations on compact sets is proved under several sampling schemes…
We study the problem of exponential mixing and large deviations for discrete-time Markov processes associated with a class of random dynamical systems. Under some dissipativity and regularisation hypotheses for the underlying deterministic…
We develop an approach for solving one-sided optimal stopping problems in discrete time for general underlying Markov processes on the real line. The main idea is to transform the problem into an auxiliary problem for the ladder height…