Related papers: Weak approximation rates for integral functionals …

A new weak bisimulation semantics is defined for Markov automata that, in addition to abstracting from internal actions, sums up the expected values of consecutive exponentially distributed delays possibly intertwined with internal actions.…

The estimation of absorption time distributions of Markov jump processes is an important task in various branches of statistics and applied probability. While the time-homogeneous case is classic, the time-inhomogeneous case has recently…

We design a particle interpretation of Feynman-Kac measures on path spaces based on a backward Markovian representation combined with a traditional mean field particle interpretation of the flow of their final time marginals. In contrast to…

The arrival time probability distribution is defined by analogy with the classical mechanics. The difficulty of requirement to have the values of non-commuting operators is circumvented using the concept of weak measurements. The proposed…

We present a systematic method for computing explicit approximations to martingale representations for a large class of Brownian functionals. The approximations are obtained by obtained by computing a directional derivative of the weak…

Our aim is to find sufficient conditions for weak convergence of stochastic integrals with respect to the state occupation measure of a Markov chain. First, we study properties of the state indicator function and the state occupation…

In this paper, we develop necessary and sufficient conditions for the validity of a martingale approximation for the partial sums of a stationary process in terms of the maximum of consecutive errors. Such an approximation is useful for…

The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…

For a countable-state Markov decision process we introduce an embedding which produces a finite-state Markov decision process. The finite-state embedded process has the same optimal cost, and moreover, it has the same dynamics as the…

The limits of scaled relative entropies between probability distributions associated with N-particle weakly interacting Markov processes are considered. The convergence of such scaled relative entropies is established in various settings.…

The simplest, and most common, stochastic model for population processes, including those from biochemistry and cell biology, are continuous time Markov chains. Simulation of such models is often relatively straightforward as there are…

An efficient discrete time and space Markov chain approximation employing a Brownian bridge correction for computing curvilinear boundary crossing probabilities for general diffusion processes was recently proposed in Liang and Borovkov…

This article presents a short and concise description of stochastic approximation algorithms in reinforcement learning of Markov decision processes. The algorithms can also be used as a suboptimal method for partially observed Markov…

We have recently defined a weak Markovian bisimulation equivalence in an integrated-time setting, which reduces sequences of exponentially timed internal actions to individual exponentially timed internal actions having the same average…

We study the long-time behavior of an additive functional that takes into account the jumps of a symmetric Markov process. This process is assumed to be observed through a biased observation scheme that includes the survival to events of…

First, sufficient conditions are given for a triangular array of random vectors such that the sequence of related random step functions converges towards a (not necessarily time homogeneous) diffusion process. These conditions are weaker…

The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…

In this paper, the weak convergence of additive functionals of processes with locally independent increments and with Markov switching in the scheme of Poisson approximation is proved. For the relative compactness, a method proposed by R.…

We use the martingale convergence method to get the weak convergence theorem on general functionals of partial sums of independent heavy-tailed random variables. The limiting process is the stochastic integral driven by $\alpha-$stable…

A powerful tool for studying long-term convergence of a Markov process to its stationary distribution is a Lyapunov function. In some sense, this is a substitute for eigenfunctions. For a stochastically ordered Markov process on the…