Related papers: Waiting times and stopping probabilities for patte…

We introduce the class of constant probability (CP) programs and show that classical results from probability theory directly yield a simple decision procedure for (positive) almost sure termination of programs in this class. Moreover,…

We study the asymptotic hitting time $\tau^{(n)}$ of a family of Markov processes $X^{(n)}$ to a target set $G^{(n)}$ when the process starts from a trap defined by very general properties. We give an explicit description of the law of…

We consider first passage times $\tau_u = \inf\{n:\; Y_n>u\}$ for the perpetuity sequence $$ Y_n = B_1 + A_1 B_2 + \cdots + (A_1\ldots A_{n-1})B_n, $$ where $(A_n,B_n)$ are i.i.d. random variables with values in ${\mathbb R} ^+\times…

Consider a system evolving according to an absorbing discrete-time Markov chain with known transition matrix. The state of the system is observed at two points in time, separated by an unknown number of generations. We are interested in…

Assume that letters (from a finite alphabet) in a text form a Markov chain. We track two distinct words, $U$ and $D$. A gambler gains 1 point for each occurrence of $U$ (including overlapping occurrences) and loses 1 point for each…

Motivated by applications in telecommunications, computer scienceand physics, we consider a discrete-time Markov process withrestart. At each step the process eitherwith a positive probability restarts from a given distribution, orwith the…

Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stopping times $\lambda_n$…

Let $X$ be the constrained random walk on $\mathbb{Z}_+^d$ $d >2$, having increments $e_1$, $-e_i+e_{i+1}$ $i=1,2,3,...,d-1$ and $-e_d$ with probabilities $\lambda$, $\mu_1$, $\mu_2$,...,$\mu_d$, where $\{e_1,e_2,..,e_d\}$ are the standard…

We present the analysis of behavior of N identical finite time memories with the imperfections characterized by a step-function model, based on a study of independent copies of the geometric distribution. We show a step-by-step derivation…

A divide-and-conquer approach to analyzing Markov chains (MCs) is not utilized as widely as it could be, despite its potential benefits. One primary reason for this is the fact that most MC decomposition approaches involve a complex and…

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…

We study a model of a polling system, that is, a collection of $d$ queues with a single server that switches from queue to queue. The service time distribution and arrival rates change randomly every time a queue is emptied. This model is…

This paper gives a method for computing distributions associated with patterns in the state sequence of a hidden Markov model, conditional on observing all or part of the observation sequence. Probabilities are computed for very general…

We prove a Chernoff-type bound for sums of matrix-valued random variables sampled via a regular (aperiodic and irreducible) finite Markov chain. Specially, consider a random walk on a regular Markov chain and a Hermitian matrix-valued…

Consider a system of \(n\) players in which each initially starts on a different team. At each time step, we select an individual winner and an individual loser randomly and the loser joins the winner's team. The resulting Markov chain and…

For a finite state Markov process and a finite collection $\{ \Gamma_k, k \in K \}$ of subsets of its state space, let $\tau_k$ be the first time the process visits the set $\Gamma_k$. We derive explicit/recursive formulas for the joint…

Let 0<\alpha<1/2. We show that the mixing time of a continuous-time reversible Markov chain on a finite state space is about as large as the largest expected hitting time of a subset of stationary measure at least \alpha of the state space.…

We use coupling to study the time taken until the distribution of a statistic on a Markov chain is close to its stationary distribution. Coupling is a common technique used to obtain upper bounds on mixing times of Markov chains, and we…

Parametric Markov chains have been introduced as a model for families of stochastic systems that rely on the same graph structure, but differ in the concrete transition probabilities. The latter are specified by polynomial constraints for…

Let $(X,\p_x)$ be a continuous time Markov chain with finite or countable state space $S$ and let $T$ be its first passage time in a subset $D$ of $S$. It is well known that if $\mu$ is a quasi-stationary distribution relatively to $T$,…