Related papers: Finiteness of hitting times under taboo
Motivated by a risk process with positive and negative premium rates, we consider a real-valued Markov additive process with finitely many background states. This additive process linearly increases or decreases while the background state…
A general criterion is given for when a Markov chain trapped with probability p(x) in state x will be almost surely trapped. The quenched (state x is a trap forever with probability p(x)) and annealed (state x traps with probability p(x) on…
An aperiodic and irreducible Markov chain on a finite state space converges to its stationary distribution. When convergence to equilibrium is measured by total variation distance, there exists an optimal coupling and a maximal coupling…
We study the computation of lower and upper probabilities of hitting a target set of states for imprecise Markov chains, where transition uncertainty is modelled by a convex set of transition matrices. In the precise case, hitting…
Quantum walks play an important role in the area of quantum algorithms. Many interesting problems can be reduced to searching marked states in a quantum Markov chain. In this context, the notion of quantum hitting time is very important,…
Let $P$ be the transition matrix of a finite, irreducible and reversible Markov chain. We say the continuous time Markov chain $X$ has transition matrix $P$ and speed $\lambda$ if it jumps at rate $\lambda$ according to the matrix $P$. Fix…
It is well known that the distributions of hitting times in Markov chains are quite irregular, unless the limit as time tends to infinity is considered. We show that nevertheless for a typical finite irreducible Markov chain and for…
For a finite state Markov process and a finite collection $\{ \Gamma_k, k \in K \}$ of subsets of its state space, let $\tau_k$ be the first time the process visits the set $\Gamma_k$. We derive explicit/recursive formulas for the joint…
We consider moments of the return times (or first hitting times) in a discrete time discrete space Markov chain. It is classical that the finiteness of the first moment of a return time of one state implies the finiteness of the first…
We consider finite irreducible Markov chains. It was shown that mean hitting time from one state to another satisfies the triangle inequality. Hence, sum of mean hitting time between couple of states in both directions is a metric on the…
Consider a Markov chain with finite state space and suppose you wish to change time replacing the integer step index $n$ with a random counting process $N(t)$. What happens to the mixing time of the Markov chain? We present a partial reply…
Given an irreducible discrete-time Markov chain on a finite state space, we consider the largest expected hitting time $T(\alpha)$ of a set of stationary measure at least $\alpha$ for $\alpha\in(0,1)$. We obtain tight inequalities among the…
In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent…
We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of…
This paper studies finite-time safety and reach-avoid verification for stochastic discrete-time dynamical systems. The aim is to ascertain lower and upper bounds of the probability that, within a predefined finite-time horizon, a system…
Suppose that $\mathcal C$ is a finite collection of patterns. Observe a Markov chain until one of the patterns in $\mathcal C$ occurs as a run. This time is denoted by $\tau$. In this paper, we aim to give an easy way to calculate the mean…
The cutoff phenomenon is an abrupt transition from out of equilibrium to equilibrium undergone by certain Markov processes in the limit where the size of the state space tends to infinity: instead of decaying gradually over time, their…
We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we…
Studying the behaviour of Markov processes at boundary points of the state space has a long history, dating back all the way to William Feller. With different motivations in mind entrance and exit questions have been explored for different…
Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously…