Related papers: Markov chains conditioned never to wait too long a…

Suppose that $\mathcal C$ is a finite collection of patterns. Observe a Markov chain until one of the patterns in $\mathcal C$ occurs as a run. This time is denoted by $\tau$. In this paper, we aim to give an easy way to calculate the mean…

By proving a local limit theorem for higher-order transitions, we determine the time required for necklace chains to be close to stationarity. Because necklace chains, built by arranging identical smaller chains around a directed cycle, are…

In a Markov chain started at a state $x$, the hitting time $\tau(y)$ is the first time that the chain reaches another state $y$. We study the probability $\mathbf{P}_x(\tau(y) = t)$ that the first visit to $y$ occurs precisely at a given…

A sequence of Markov chains is said to exhibit (total variation) cutoff if the convergence to stationarity in total variation distance is abrupt. We consider reversible lazy chains. We prove a necessary and sufficient condition for the…

For a class of irreducible Markov chains with an infinitely countable set of states, we establish a new verifiable necessary and sufficient condition for recurrence and transience. We show that if one of the basic assumptions is not…

Given an irreducible discrete-time Markov chain on a finite state space, we consider the largest expected hitting time $T(\alpha)$ of a set of stationary measure at least $\alpha$ for $\alpha\in(0,1)$. We obtain tight inequalities among the…

We consider Markov chains that obey the following general non-linear state space model: $\Phi_{k+1} = F(\Phi_k, \alpha(\Phi_k, U_{k+1}))$ where the function $F$ is $C^1$ while $\alpha$ is typically discontinuous and $\{U_k: k \in…

It is well known that the distributions of hitting times in Markov chains are quite irregular, unless the limit as time tends to infinity is considered. We show that nevertheless for a typical finite irreducible Markov chain and for…

We consider a continuous-time Markov chain with a finite or countable state space. For a site y and subset H of the state space, the hitting time of y under taboo H is defined to be infinite if the process trajectory hits H before y, and…

We consider a Markov chain $(M_{n})_{n\ge 0}$ on the set $\mathbb{N}_{0}$ of nonnegative integers which is eventually decreasing, i.e. $\mathbb{P}\{M_{n+1}<M_{n}|M_{n}\ge a\}=1$ for some $a\in\mathbb{N}$ and all $n\ge 0$. We are interested…

Let X be a continuous-time Markov chain in a finite set I, let h be a mapping of I onto another set, and let Y be defined by Y_t=h(X_t), (for t nonnegative). We address the filtering problem for X in terms of the observation Y, which is not…

Consider a Markov chain $(X_n)_{n\geqslant 0}$ with values in the state space $\mathbb X$. Let $f$ be a real function on $\mathbb X$ and set $S_0=0,$ $S_n = f(X_1)+\cdots + f(X_n),$ $n\geqslant 1$. Let $\mathbb P_x$ be the probability…

Consider a parametric statistical model $P(\mathrm{d}x|\theta)$ and an improper prior distribution $\nu(\mathrm{d}\theta)$ that together yield a (proper) formal posterior distribution $Q(\mathrm{d}\theta|x)$. The prior is called strongly…

For each $n$ let $Y^n_t$ be a continuous time symmetric Markov chain with state space $n^{-1} \Z^d$. A condition in terms of the conductances is given for the convergence of the $Y^n_t$ to a symmetric Markov process $Y_t$ on $\R^d$. We have…

Let $(X_n \colon n\in\Z)$ be a two-sided recurrent Markov chain with fixed initial state $X_0$ and let $\nu$ be a probability measure on its state space. We give a necessary and sufficient criterion for the existence of a non-randomized…

Let 0<\alpha<1/2. We show that the mixing time of a continuous-time reversible Markov chain on a finite state space is about as large as the largest expected hitting time of a subset of stationary measure at least \alpha of the state space.…

We study the asymptotic hitting time $\tau^{(n)}$ of a family of Markov processes $X^{(n)}$ to a target set $G^{(n)}$ when the process starts from a trap defined by very general properties. We give an explicit description of the law of…

In this paper we consider stopping problems for continuous-time Markov chains under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. More precisely our aim is to maximize the certainty…

We introduce and analyze a long-memory continuous-time Markov chain on $\mathbb{R}_{+}$ whose jump mechanism depends explicitly on a state in the past. From the present state $x_0$, the process jumps to $x_0\left(1+\frac{1}{N}\right)$ or…

A general criterion is given for when a Markov chain trapped with probability p(x) in state x will be almost surely trapped. The quenched (state x is a trap forever with probability p(x)) and annealed (state x traps with probability p(x) on…