Related papers: A transient Markov chain with finitely many cutpoi…
We study random walk on complex networks with transition probabilities which depend on the current and previously visited nodes. By using an absorbing Markov chain we derive an exact expression for the mean first passage time between pairs…
We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$.…
The divergence of a group is a quasi-isometry invariant defined in terms of pairs of points and lengths of paths avoiding a suitable ball around the identity. In this paper we study "random divergence'', meaning the divergence at two points…
We revisit the so-called Cat-and-Mouse Markov chain, studied earlier by Litvak and Robert (2012). This is a 2-dimensional Markov chain on the lattice $\mathbb{Z}^2$, where the first component (the cat) is a simple random walk and the second…
We discuss a couple of examples of Markov chains. This note is written primarily for school students; it is based on a lecture given by the first author at a Math Circle at NAS (www.assagames.com/nas).
A discrete time branching process where the offspring distribution is generation-dependent, and the number of reproductive individuals is controlled by a random mechanism is considered. This model is a Markov chain but, in general, the…
Quantum trajectories are Markov processes modeling the evolution of a quantum system subjected to repeated independent measurements. Inspired by the theory of random products of matrices, it has been shown that these Markov processes admit…
The Transience objective is not to visit any state infinitely often. While this is not possible in finite Markov Decision Process (MDP), it can be satisfied in countably infinite ones, e.g., if the transition graph is acyclic. We prove the…
An aperiodic and irreducible Markov chain on a finite state space converges to its stationary distribution. When convergence to equilibrium is measured by total variation distance, there exists an optimal coupling and a maximal coupling…
This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…
We consider topological Markov chains (also called Markov shifts) on countable graphs. We show that a transient graph can be extended to a recurrent graph of equal entropy which is either positive recurrent of null recurrent, and we give an…
R\'emy's algorithm is a Markov chain that iteratively generates a sequence of random trees in such a way that the $n^{\mathrm{th}}$ tree is uniformly distributed over the set of rooted, planar, binary trees with $2n+1$ vertices. We obtain a…
A branching process in a Markovian environment consists of an irreducible Markov chain on a set of "environments" together with an offspring distribution for each environment. At each time step the chain transitions to a new random…
The rotor walk on a graph is a deterministic analogue of random walk. Each vertex is equipped with a rotor, which routes the walker to the neighbouring vertices in a fixed cyclic order on successive visits. We consider rotor walk on an…
We study a colored generalization of the famous simple-switch Markov chain for sampling the set of graphs with a fixed degree sequence. Here we consider the space of graphs with colored vertices, in which we fix the degree sequence and…
Consider the barycentric subdivision which cuts a given triangle along its medians to produce six new triangles. Uniformly choosing one of them and iterating this procedure gives rise to a Markov chain. We show that almost surely, the…
In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we…
In this paper, we study dynamical properties as hypercyclicity, supercyclicity, frequent hypercyclicity and chaoticity for transition operators associated to countable irreductible Markov chains. As particular cases, we consider simple…
We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of…
We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…