Related papers: Markov Processes with Restart
This paper deals with ergodic theorems for particular time-inhomogeneous Markov processes, whose the time-inhomogeneity is asymptotically periodic. Under a Lyapunov/minorization condition, it is shown that, for any measurable bounded…
By modeling the interaction of a system with an environment through a renewal approach, we demonstrate that completely positive non-Markovian dynamics may develop some unexplored non-standard statistical properties. The renewal approach is…
We consider Markov decision processes with synchronizing objectives, which require that a probability mass of $1-\epsilon$ accumulates in a designated set of target states, either once, always, infinitely often, or always from some point…
This paper presents a stochastic model motivated by the study of a virus-like evolving population with different mutation rates. This is a continuous time birth-death model: the birth processes are mutually-exciting Hawkes processes and the…
We obtain the posterior distribution of a random process conditioned on observing the empirical frequencies of a finite sample path. We find under a rather broad assumption on the "dependence structure" of the process, {\em c.f.}…
Let {X_n,n\geq0} be a Markov chain on a general state space X with transition probability P and stationary probability \pi. Suppose an additive component S_n takes values in the real line R and is adjoined to the chain such that…
Reciprocal processes are acausal generalizations of Markov processes introduced by Bernstein in 1932. In the literature, a significant amount of attention has been focused on developing dynamical models for reciprocal processes. In this…
A method is proposed to reconstruct a cyclic time-inhomogeneous Markov pro- cess from measured data. First, a time-inhomogeneous Markov model is fit to the data, taken here from measurements on a wind turbine. From the time-dependent…
The marked Hawkes risk process is a compound point process for which the occurrence and amplitude of past events impact the future. Thanks to its autoregressive properties, it found applications in various fields such as neuosciences,…
In this work, we are concerned with existence and uniqueness of invariant measures for path-dependent random diffusions and their time discretizations. The random diffusion here means a diffusion process living in a random environment…
Consider the continuous-time Markov Branching Process. In critical case we consider a situation when the generating function of intensity of transformation of particles has the infinite second moment, but its tail regularly varies in sense…
The use of coordinate processes for the modelling of impulse control for general Markov processes typically involves the construction of a probability measure on a countable product of copies of the path space. In addition, admissibility of…
We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…
We show the variational convergence of an irreversible Markov jump process describing a finite stochastic particle system to the solution of a countable infinite system of deterministic time-inhomogeneous quadratic differential equations…
We investigate the effects of markovian resseting events on continuous time random walks where the waiting times and the jump lengths are random variables distributed according to power law probability density functions. We prove the…
We consider the first exit time of a nonnegative Harris-recurrent Markov process from the interval $[0,A]$ as $A\to\infty$. We provide an alternative method of proof of asymptotic exponentiality of the first exit time (suitably…
The estimation of absorption time distributions of Markov jump processes is an important task in various branches of statistics and applied probability. While the time-homogeneous case is classic, the time-inhomogeneous case has recently…
We examine a constrained Markov decision process under uncertain transition probabilities, with the uncertainty modeled as deviations from observed transition probabilities. We construct the uncertainty set associated with the deviations…
Semi-Markov processes generalize Markov processes by adding temporal memory effects as expressed by a semi-Markov kernel. We recall the path weight for a semi-Markov trajectory and the fact that thermodynamic consistency in equilibrium…
Markov decision processes (MDP) are a well-established model for sequential decision-making in the presence of probabilities. In robust MDP (RMDP), every action is associated with an uncertainty set of probability distributions, modelling…