Related papers: Conditioning continuous-time Markov processes by g…
Diffusive approximations of Markov jump processes often fail to accurately capture large fluctuations. This is confounding, as the rare events triggered by these large fluctuations, such as the failure of electronic memories, are often the…
Let $R$ be a continuous-time Markov process on the time interval $[0,1]$ with values in some state space $X$. We transform this reference process $R$ into $P:=f(X_0)\exp (-\int_0^1 V_t(X_t) dt) g(X_1)\,R$ where $f,g$ are nonnegative…
We are interested in understanding stability (almost sure boundedness) of stochastic approximation algorithms (SAs) driven by a `controlled Markov' process. Analyzing this class of algorithms is important, since many reinforcement learning…
In this paper, we establish a version of the central limit theorem for Markov-Feller continuous time processes (with a Polish state space) that are exponentially ergodic in the bounded-Lipschitz distance and enjoy a continuous form of the…
Semi-Markov processes are Markovian processes in which the firing time of the transitions is modelled by probabilistic distributions over positive reals interpreted as the probability of firing a transition at a certain moment in time. In…
Consider a continuous time particle system $\eta^t=(\eta^t(k),k\in \mathbb{L})$, indexed by a lattice $\mathbb{L}$ which will be either $\mathbb{Z}$, $\mathbb{Z}/n\mathbb{Z}$, a segment $\{1,\cdots, n\}$, or $\mathbb{Z}^d$, and taking its…
Conditioning Markov processes to avoid a domain is a classical problem that has been studied in many settings. Ingredients for standard arguments involve the leading order tail asymptotics of the distribution of the first hitting time of…
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In…
When the unconditioned process is a diffusion living on the half-line $x \in ]-\infty,a[$ in the presence of an absorbing boundary condition at position $x=a$, we construct various conditioned processes corresponding to finite or infinite…
We consider sequences $(X_t^N)_{t\geq0}$ of Markov processes in two dimensions whose fluid limit is a stable solution of an ordinary differential equation of the form $\dot{x}_t=b(x_t)$, where $b(x)={\pmatrix{-\mu 0 0 \lambda}}x+\tau(x)$…
Hybrid systems whose mode dynamics are governed by non-linear ordinary differential equations (ODEs) are often a natural model for biological processes. However such models are difficult to analyze. To address this, we develop a…
The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…
The fluctuation-dissipation theorem is a central result in statistical mechanics and is usually formulated for systems described by diffusion processes. In this paper, we propose a generalization for a wider class of stochastic processes,…
We establish sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the existence and exponential ergodicity of the Q-process, the process…
We consider the task of generating draws from a Markov jump process (MJP) between two time-points at which the process is known. Resulting draws are typically termed bridges and the generation of such bridges plays a key role in…
Continuous-time Markov decision processes are an important class of models in a wide range of applications, ranging from cyber-physical systems to synthetic biology. A central problem is how to devise a policy to control the system in order…
A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…
Density dependent families of Markov chains, such as the stochastic models of mass-action chemical kinetics, converge for large values of the indexing parameter $N$ to deterministic systems of differential equations (Kurtz, 1970). Moreover…
In the presence of quantum measurements with direct photon detection the evolution of open quantum systems is usually described by stochastic master equations with jumps. Heuristically, from these equations one can obtain diffusion models…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…