Related papers: Finiteness of hitting times under taboo
The safety of infinite state systems can be checked by a backward reachability procedure. For certain classes of systems, it is possible to prove the termination of the procedure and hence conclude the decidability of the safety problem.…
We revisit the symbolic verification of Markov chains with respect to finite horizon reachability properties. The prevalent approach iteratively computes step-bounded state reachability probabilities. By contrast, recent advances in…
Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…
We suggest an approach to obtaining general two-sided bounds on the rate of convergence in terms of special "weighted" norms related to total variation. Some important classes of continuous-time Markov chains are considered:…
The first hitting times of a stochastic process, i.e., the first time a process reaches a particular level, are of significant interest across various scientific disciplines, including biology, chemistry, and economics. We modify the…
he starting process with countable number of types \mu(t) generates a stopped branching process \xi(t). The starting process stops, by falling into the nonempty set S. It is assumed, that the starting process is subcritical, indecomposable…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
Given an infinitesimal perturbation of a discrete-time finite Markov chain, we seek the states that are stable despite the perturbation, \textit{i.e.} the states whose weights in the stationary distributions can be bounded away from $0$ as…
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…
We consider Markov chains that obey the following general non-linear state space model: $\Phi_{k+1} = F(\Phi_k, \alpha(\Phi_k, U_{k+1}))$ where the function $F$ is $C^1$ while $\alpha$ is typically discontinuous and $\{U_k: k \in…
In this paper, we are interested in investigating the perturbation bounds for the stationary distributions for discrete-time or continuous-time Markov chains on a countable state space. For discrete-time Markov chains, two new norm-wise…
Let $(X,\p_x)$ be a continuous time Markov chain with finite or countable state space $S$ and let $T$ be its first passage time in a subset $D$ of $S$. It is well known that if $\mu$ is a quasi-stationary distribution relatively to $T$,…
The objective of this work is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite-time horizon discounted cost. The continuous-time controlled…
We give an example of a transient reversible Markov chain that almost surely has only a finite number of cutpoints. We explain how this is relevant to a conjecture of Diaconis and Freedman and a question of Kaimanovich. We also answer…
Consider a Markov chain with finite state $\{0, 1, ..., d\}$. We give the generation functions (or Laplace transforms) of absorbing (passage) time in the following two situations : (1) the absorbing time of state $d$ when the chain starts…
Consider a finite number of balls initially placed in $L$ bins. At each time step a ball is taken from each non-empty bin. Then all the balls are uniformly reassigned into bins. This finite Markov chain is called Repeated Balls-into-Bins…
Starting from an n-by-n matrix of zeros, choose uniformly random zero entries and change them to ones, one-at-a-time, until the matrix becomes invertible. We show that with probability tending to one as n tends to infinity, this occurs at…
We consider irreducible Markov chains on a finite state space. We show that the mixing time of any such chain is equivalent to the maximum, over initial states $x$ and moving large sets $(A_s)_s$, of the hitting time of $(A_s)_s$ starting…
In this paper, we study finite-time ruin probabilities for the compound Markov binomial risk model - a discrete-time model where claim sizes are modulated by a finite-state ergodic Markov chain. In the classic (non-modulated) case, the risk…
We consider Markov processes in continuous time with state space $\posint^N$ and provide two sufficient conditions and one necessary condition for the existence of moments $E(\|X(t)\|^r)$ of all orders $r \in \nat$ for all $t \geq 0$. The…