Related papers: Surprise probabilities in Markov chains

We consider first passage times $\tau_u = \inf\{n:\; Y_n>u\}$ for the perpetuity sequence $$ Y_n = B_1 + A_1 B_2 + \cdots + (A_1\ldots A_{n-1})B_n, $$ where $(A_n,B_n)$ are i.i.d. random variables with values in ${\mathbb R} ^+\times…

We consider random walks in the form of nearest-neighbor hopping on Erdos-Renyi random graphs of finite fixed mean degree c as the number of vertices N tends to infinity. In this regime, using statistical field theory methods, we develop an…

We study nonconcentration of hitting times for simple random walk on finite graphs. We prove that, for every connected graph with $n$ vertices, \[ \operatorname{Var}_x(\tau_y)+\mathbb E_x\tau_y \ge \frac{(\mathbb E_x\tau_y)^2}{1+\log n}, \]…

We show that the distribution of the first return time $\tau$ to the origin, v, of a simple random walk on an infinite recurrent graph is heavy tailed and nonconcentrated. More precisely, if $d_v$ is the degree of v, then for any $t\geq1$…

It is well known that the distributions of hitting times in Markov chains are quite irregular, unless the limit as time tends to infinity is considered. We show that nevertheless for a typical finite irreducible Markov chain and for…

We study the problem of generating a sample from the stationary distribution of a Markov chain, given a method to simulate the chain. We give an approximation algorithm for the case of a random walk on a regular graph with n vertices that…

In the investigation of limits of Markov chains, the presence of states which become instantaneous states in the limit may prevent the convergence of the chain in the Skorohod topology. We present in this article a weaker topology adapted…

Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…

This paper presents a novel theoretical Monte Carlo Markov chain procedure in the framework of graphs. It specifically deals with the construction of a Markov chain whose empirical distribution converges to a given reference one. The Markov…

We study the computation of lower and upper probabilities of hitting a target set of states for imprecise Markov chains, where transition uncertainty is modelled by a convex set of transition matrices. In the precise case, hitting…

We consider an elementary model for self-organised criticality, the activated random walk on the complete graph. We introduce a discrete time Markov chain as follows. At each time step, we add an active particle at a random vertex and let…

Consider a random walk on a tree $G=(V,E)$. For $v,w \in V$, let the hitting time $H(v,w)$ denote the expected number of steps required for the random walk started at $v$ to reach $w$, and let $\pi_v = \mathrm{deg}(v)/2|E|$ denote the…

Markov chains are a convenient means of generating realizations of networks, since they require little more than a procedure for rewiring edges. If a rewiring procedure exists for generating new graphs with specified statistical properties,…

A Markov chain is considered whose states are orderings of an underlying fixed tree and whose transitions are local "random-to-front" reorderings, driven by a probability distribution on subsets of the leaves. The eigenvalues of the…

Let 0<\alpha<1/2. We show that the mixing time of a continuous-time reversible Markov chain on a finite state space is about as large as the largest expected hitting time of a subset of stationary measure at least \alpha of the state space.…

We prove the first Chernoff-Hoeffding bounds for general nonreversible finite-state Markov chains based on the standard L_1 (variation distance) mixing-time of the chain. Specifically, consider an ergodic Markov chain M and a weight…

A random walk on a $N$-dimensional hypercube is a discrete time stochastic process whose state space is the set $\{-1,+1\}^{N}$, which has uniform probability of reaching any neighbour state, and probability zero of reaching a non-neighbour…

An analytical formula for the occurence probability of Markovian stochastic paths with repeatedly visited and/or equal departure rates is derived. This formula is essential for an efficient investigation of the trajectories belonging to…

We study convergence to equilibrium for a large class of Markov chains in random environment. The chains are sparse in the sense that in every row of the transition matrix $P$ the mass is essentially concentrated on few entries. Moreover,…

Markov chains are convenient means of generating realizations of networks with a given (joint or otherwise) degree distribution, since they simply require a procedure for rewiring edges. The major challenge is to find the right number of…