Related papers: Finiteness of hitting times under taboo
The cover time of a Markov chain on a finite state space is the expected time until all states are visited. We show that if the cover time of a discrete-time Markov chain with rational transitions probabilities is bounded, then it is a…
We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which…
We study a catalytic branching process (CBP) with any finite set of catalysts. This model describes a system of particles where the movement is governed by a Markov chain with arbitrary finite or countable state space and the branching may…
In a Markov chain started at a state $x$, the hitting time $\tau(y)$ is the first time that the chain reaches another state $y$. We study the probability $\mathbf{P}_x(\tau(y) = t)$ that the first visit to $y$ occurs precisely at a given…
For a symmetric, homogeneous and irreducible random walk on d-dimensional integer lattice Z^d, having zero mean and a finite variance of jumps, we study the passage times (with possible infinite values) determined by the starting point x,…
Let 0<\alpha<1/2. We show that the mixing time of a continuous-time reversible Markov chain on a finite state space is about as large as the largest expected hitting time of a subset of stationary measure at least \alpha of the state space.…
Motivated by applications in telecommunications, computer scienceand physics, we consider a discrete-time Markov process withrestart. At each step the process eitherwith a positive probability restarts from a given distribution, orwith the…
In the present paper, we investigate the relationship between hitting times and hitting probabilities in discrete-time imprecise Markov chains (IMCs). We define lower and upper hitting times and probabilities for IMCs whose set of…
We present a new algorithm for the statistical model checking of Markov chains with respect to unbounded temporal properties, such as reachability and full linear temporal logic. The main idea is that we monitor each simulation run on the…
Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as…
We study the hitting times of Markov processes to target set $G$, starting from a reference configuration $x_0$ or its basin of attraction. The configuration $x_0$ can correspond to the bottom of a (meta)stable well, while the target $G$…
Consider the continuous-time Markov Branching Process. In critical case we consider a situation when the generating function of intensity of transformation of particles has the infinite second moment, but its tail regularly varies in sense…
Motivated by Feller's coin-tossing problem, we consider the problem of conditioning an irreducible Markov chain never to wait too long at 0. Denoting by $\tau$ the first time that the chain, $X$, waits for at least one unit of time at the…
We consider the problem of characterising expected hitting times and hitting probabilities for imprecise Markov chains. To this end, we consider three distinct ways in which imprecise Markov chains have been defined in the literature: as…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…
We consider irreducible reversible discrete time Markov chains on a finite state space. Mixing times and hitting times are fundamental parameters of the chain. We relate them by showing that the mixing time of the lazy chain is equivalent…
Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
We propose a new definition of metastability of Markov processes on countable state spaces. We obtain sufficient conditions for a sequence of processes to be metastable. In the reversible case these conditions are expressed in terms of the…
We introduce a Markov chain model of concurrent quantum programs. This model is a quantum generalization of Hart, Sharir and Pnueli's probabilistic concurrent programs. Some characterizations of the reachable space, uniformly repeatedly…