Related papers: Convergence of a Second Order Markov Chain

All one-condition generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain, can be uniquely specified in terms of the stationary probabilities and the mean first passage times…

We consider consistent particle systems, which include independent random walkers, the symmetric exclusion and inclusion processes, as well as the dual of the KMP model. Consistent systems are such that the distribution obtained by first…

In this paper, we first extend the celebrated PageRank modification to a higher-order Markov chain. Although this system has attractive theoretical properties, it is computationally intractable for many interesting problems. We next study a…

We examine two analytical characterisation of the metastable behavior of a Markov chain. The first one expressed in terms of its transition probabilities, and the second one in terms of its large deviations rate functional. Consider a…

We introduce a framework to approximate a Markov Decision Process that stands on two pillars: state aggregation -- as the algorithmic infrastructure; and central-limit-theorem-type approximations -- as the mathematical underpinning of…

The analysis of many problems of interest associated with Markov chains, e.g. stationary distributions, moments of first passage time distributions and moments of occupation time random variables, involves the solution of a system of linear…

We give recurrence and transience criteria for two cases of time-homogeneous Markov chains on the real line with transition kernel $p(x,dy)=f_x(y-x)dy$, where $f_x(y)$ are probability densities of symmetric distributions and, for large…

Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the…

We study long time behavior of a discrete time weakly interacting particle system, and the corresponding nonlinear Markov process in $\mathbb{R}^d$, described in terms of a general stochastic evolution equation. In a setting where the state…

We establish a new Bernstein-type deviation inequality for general (non-reversible) discrete-time Markov chains via an elementary approach. More robust than existing works in the literature, our result only requires the Markov chain to…

Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…

This paper is about the rate of convergence of the Markov chain $X_{n+1}=AX_{n}+B_{n}$ (mod $p$), where $A$ is an integer matrix with nonzero eigenvalues and ${B_{n}}_{n}$ is a sequence of independent and identically distributed integer…

In the first part of this paper we generalize the result of Georgiou-Pavon that a positive square matrix can be scaled uniquely to a column stochastic matrix which maps a given positive probability vector to another given positive…

Let {X_n,n\geq0} be a Markov chain on a general state space X with transition probability P and stationary probability \pi. Suppose an additive component S_n takes values in the real line R and is adjoined to the chain such that…

We study an irreducible Markov chain on the category of finite abelian $p$-groups, whose stationary measure is the Cohen-Lenstra distribution. This Markov chain arises when one studies the cokernel of a random matrix $M$, after conditioning…

We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these…

In this paper, we consider a general class of two-time-scale Markov chains whose transition rate matrix depends on a parameter $\lambda>0$. We assume that some transition rates of the Markov chain will tend to infinity as…

We consider moments of the return times (or first hitting times) in a discrete time discrete space Markov chain. It is classical that the finiteness of the first moment of a return time of one state implies the finiteness of the first…

This paper presents a novel theoretical Monte Carlo Markov chain procedure in the framework of graphs. It specifically deals with the construction of a Markov chain whose empirical distribution converges to a given reference one. The Markov…

We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…