Related papers: Reduction of Markov chains with two-time-scale sta…

This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when the chain enters into a subset A, then sample another subset B far away from A and end when the chain again return to A. The…

We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…

We introduce a general class of distances (metrics) between Markov chains, which are based on linear behaviour. This class encompasses distances given topologically (such as the total variation distance or trace distance) as well as by…

We study the approximation of a Markov chain on a reduced state space, for both discrete- and continuous-time Markov chains. In this context, we extend the existing theory of formal error bounds for the approximated transient distributions.…

For each $n$ let $Y^n_t$ be a continuous time symmetric Markov chain with state space $n^{-1} \Z^d$. A condition in terms of the conductances is given for the convergence of the $Y^n_t$ to a symmetric Markov process $Y_t$ on $\R^d$. We have…

Let $(X_t)_{t = 0 }^{\infty}$ be an irreducible reversible discrete time Markov chain on a finite state space $\Omega $. Denote its transition matrix by $P$. To avoid periodicity issues (and thus ensuring convergence to equilibrium) one…

We investigate the problem of quantifying contraction coefficients of Markov transition kernels in Kantorovich ($L^1$ Wasserstein) distances. For diffusion processes, relatively precise quantitative bounds on contraction rates have recently…

This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…

Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {P_n} be a sequence of transition matrices on a…

The distribution of the "mixing time" or the "time to stationarity" in a discrete time irreducible Markov chain, starting in state i, can be defined as the number of trials to reach a state sampled from the stationary distribution of the…

Perturbation theory for Markov chains addresses the question how small differences in the transitions of Markov chains are reflected in differences between their distributions. We prove powerful and flexible bounds on the distance of the…

This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic…

We examine two analytical characterisation of the metastable behavior of a Markov chain. The first one expressed in terms of its transition probabilities, and the second one in terms of its large deviations rate functional. Consider a…

The mean time taken by an irreducible Markov chain on a finite state space to hit a target chosen at random according to the stationary distribution does not depend on the initial state of the chain. This mean time is known as Kemeny's…

In this paper, we study a notion of local stationarity for discrete time Markov chains which is useful for applications in statistics. In the spirit of some locally stationary processes introduced in the literature, we consider triangular…

A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…

In this paper, we study consistent and partially exchangeable sequences of Markov chains on a finite state space. We provide a characterisation of the admissible transition rates via a decomposition into individual and coordinated motion of…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

We consider symmetric Markov chains on $\Bbb Z^d$ where we do {\bf not} assume that the conductance between two points must be zero if the points are far apart. Under a uniform second moment condition on the conductances, we obtain upper…

This note presents conjectures on polynomial/algebraic/sub-exponential convergence of transition probabilities for $\lambda$-null recurrent and $\lambda$-transient Markov chains in continuous time. The only known positive examples are in…