Related papers: Walks on chains

Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…

We develop a general theory of Markov chains realizable as random walks on $\mathscr R$-trivial monoids. It provides explicit and simple formulas for the eigenvalues of the transition matrix, for multiplicities of the eigenvalues via…

We introduce a general class of distances (metrics) between Markov chains, which are based on linear behaviour. This class encompasses distances given topologically (such as the total variation distance or trace distance) as well as by…

The main topic of these notes are Markov loops, studied in the context of continuous time Markov chains on discrete state spaces. We refer to [1] and [2] for the short "history" of the subject. In contrast with these references, symmetry is…

In this paper, we provide a methodology for computing the probability distribution of sojourn times for a wide class of Markov chains. Our methodology consists in writing out linear systems and matrix equations for generating functions…

We study Markov chains on a lattice in a codimension-one stratified independent random environment, exploiting results established in [2]. First of all the random walk is transient in dimension at least three. Focusing on dimension two,…

General characterizations of ergodic Markov chains have been developed in considerable detail. In this paper, we study the transience for discrete-time Markov chains on general state spaces, including the geometric transience and algebraic…

Consider a real hyperplane arrangement and let $\mathcal{C}$ denote the occurring chambers. Bidigare, Hanlon and Rockmore introduced a Markov chain on $\mathcal{C}$ which is a generalization of some card shuffling models used in computer…

We give an example of a transient reversible Markov chain that almost surely has only a finite number of cutpoints. We explain how this is relevant to a conjecture of Diaconis and Freedman and a question of Kaimanovich. We also answer…

This article is a tutorial on Markov chain Monte Carlo simulations and their statistical analysis. The theoretical concepts are illustrated through many numerical assignments from the author's book on the subject. Computer code (in Fortran)…

Lecture notes (in French) of a master 2 level course in applied mathematics. Contents: Part I. Markov chains on a countable space. 1. Examples 2. Summary of basic properties. 3. Spectral theory and speed of convergence. 4. Lyapunov…

Motivated by applications in Markov chain Monte Carlo, we discuss what it means for one Markov chain to be an approximation to another. Specifically included in that discussion are situations in which a Markov chain with continuous state…

Markov chains are an important example for a course on stochastic processes because simple board games can be used to illustrate the fundamental concepts. For example, a looping board game (like Monopoly) consists of all recurrent states,…

Random walks (or Markov chains) are models extensively used in theoretical computer science. Several tools, including analysis of quantities such as hitting and mixing times, are helpful for devising randomized algorithms. A notable example…

We investigate the quantum versions of a one-dimensional random walk, whose corresponding Markov Chain is of order 2. This corresponds to the walk having a memory of up to two previous steps. We derive the amplitudes and probabilities for…

We study random walk on complex networks with transition probabilities which depend on the current and previously visited nodes. By using an absorbing Markov chain we derive an exact expression for the mean first passage time between pairs…

A rigorous and largely self-contained account of (a) the bread-and-butter concepts and techniques in Markov chain theory and (b) the long-term behaviour of chains. As much as possible, the treatment is probabilistic instead of analytical (I…

The question of recurrence and transience of branching Markov chains is more subtle than for ordinary Markov chains; they can be classified in transience, weak recurrence, and strong recurrence. We review criteria for transience and weak…

The rotor walk is a derandomized version of the random walk on a graph. On successive visits to any given vertex, the walker is routed to each of the neighboring vertices in some fixed cyclic order, rather than to a random sequence of…

These are the notes for the minicourse on Markov chains delivered at the Saint Petersburg Summer School, June 2012. The main emphasis is on methods for estimating mixing times (for finite chains) and escape rates (for infinite chains).…