Related papers: Rapid mixing in Markov chains

We study inhomogeneous continuous-time weakly ergodic Markov chains with a finite state space. We introduce the notion of a Markov chain with the regular structure of an infinitesimal matrix and study the sharp upper bounds on the rate of…

The random numbers driving Markov chain Monte Carlo (MCMC) simulation are usually modeled as independent U(0,1) random variables. Tribble [Markov chain Monte Carlo algorithms using completely uniformly distributed driving sequences (2007)…

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…

Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {P_n} be a sequence of transition matrices on a…

The problem of efficiently sampling from a set of (undirected, or directed) graphs with a given degree sequence has many applications. One approach to this problem uses a simple Markov chain, which we call the switch chain, to perform the…

It is shown that the combinatorics of commutation relations is well suited for analyzing the convergence rate of certain Markov chains. Examples studied include random walk on irreducible representations, a local random walk on partitions…

Perfect sampling is a technique that uses coupling arguments to provide a sample from the stationary distribution of a Markov chain in a finite time without ever computing the distribution. This technique is very efficient if all the events…

We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…

We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…

This paper presents a simple algorithm to check whether reachability probabilities in parametric Markov chains are monotonic in (some of) the parameters. The idea is to construct - only using the graph structure of the Markov chain and…

We consider stochastic optimization problems where data is drawn from a Markov chain. Existing methods for this setting crucially rely on knowing the mixing time of the chain, which in real-world applications is usually unknown. We propose…

In networking applications, one often wishes to obtain estimates about the number of objects at different parts of the network (e.g., the number of cars at an intersection of a road network or the number of packets expected to reach a node…

Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the…

Given an infinitesimal perturbation of a discrete-time finite Markov chain, we seek the states that are stable despite the perturbation, \textit{i.e.} the states whose weights in the stationary distributions can be bounded away from $0$ as…

We discuss the efficient computation of performance, reliability, and availability measures for Markov chains; these metrics, and the ones obtained by combining them, are often called performability measures. We show that this computational…

This note studies monotone Markov chains, a subclass of Markov chains with extensive applications in operations research and economics. While the properties that ensure the global stability of these chains are well studied, their…

We consider tilings of $\mathbb{Z}^2$ by two types of squares. We are interested in the rate of convergence to the stationarity of a natural Markov chain defined for square tilings. The rate of convergence can be represented by the mixing…

We consider irreducible reversible discrete time Markov chains on a finite state space. Mixing times and hitting times are fundamental parameters of the chain. We relate them by showing that the mixing time of the lazy chain is equivalent…

We propose a method to sample stationary properties of solutions of stochastic differential equations, which is accurate and efficient if there are rarely visited regions or rare transitions between distinct regions of the state space. The…

A sequence of real numbers (x_n) is Benford if the significands, i.e. the fraction parts in the floating-point representation of (x_n) are distributed logarithmically. Similarly, a discrete-time irreducible and aperiodic finite-state Markov…