Related papers: Convergence of a Second Order Markov Chain

We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which…

This paper presents an elementary proof of stochastic stability of a discrete-time reversible Markov chain starting from a Foster-Lyapunov drift condition. Besides its relative simplicity, there are two salient features of the proof: (i) it…

A theory of systems with long-range correlations based on the consideration of binary N-step Markov chains is developed. In our model, the conditional probability that the i-th symbol in the chain equals zero (or unity) is a linear function…

We consider a diffusion given by a small noise perturbation of a dynamical system driven by a potential function with a finite number of local minima. The classical results of Freidlin and Wentzell show that the time this diffusion spends…

We study a stochastic particle system which is motivated from grain boundary coarsening in two-dimensional networks. Each particles lives on the positive real line and is labeled as belonging to either Species 1 or Species 2. Species 1…

We develop a forward-reverse EM (FREM) algorithm for estimating parameters that determine the dynamics of a discrete time Markov chain evolving through a certain measurable state space. As a key tool for the construction of the FREM method…

We consider a Markov chain $(x_n)$ whose kernel is indexed by a scaling parameter $\gamma>0$, refered to as the step size. The aim is to analyze the behavior of the Markov chain in the doubly asymptotic regime where $n\to\infty$ then…

We present a form of stratified MCMC algorithm built with non-reversible stochastic dynamics in mind. It can also be viewed as a generalization of the exact milestoning method, or form of NEUS. We prove convergence of the method under…

This paper is divided into two parts. The first part reviews the formulae for f-divergences in the study of continuous-time Markov processes and explores their applications in areas such as stochastic stability, the second law of…

Thermalization is one of the most important phenomena in statistical physics. Often, the transition probabilities between different states in the phase space is or can be approximated by constants. In this case, the system can be described…

We derive some key extremal features for $k$th order Markov chains that can be used to understand how the process moves between an extreme state and the body of the process. The chains are studied given that there is an exceedance of a…

A general criterion is given for when a Markov chain trapped with probability p(x) in state x will be almost surely trapped. The quenched (state x is a trap forever with probability p(x)) and annealed (state x traps with probability p(x) on…

We will represent the so-called Perron-Frobenius eigenvector (if exists) for infinite non-negative matrix $A$ and Metzler matrix by using its corresponding Markov chain with probability transition function.

We obtain universal estimates on the convergence to equilibrium and the times of coupling for continuous time irreducible reversible finite-state Markov chains, both in the total variation and in the L^2 norms. The estimates in total…

It is proven that the eigenvalue process of Dyson's random matrix process of size two becomes non-Markov if the common coefficient $1/\sqrt{2}$ in the non-diagonal entries is replaced by a different positive number.

We study a colored generalization of the famous simple-switch Markov chain for sampling the set of graphs with a fixed degree sequence. Here we consider the space of graphs with colored vertices, in which we fix the degree sequence and…

For a positive self-similar Markov process, X, we construct a local time for the random set, $\Theta$, of times where the process reaches its past supremum. Using this local time we describe an exit system for the excursions of X out of its…

We study scaling limits of non-increasing Markov chains with values in the set of non-negative integers, under the assumption that the large jump events are rare and happen at rates that behave like a negative power of the current state. We…

We study the so-called two-time-scale stochastic approximation, a simulation-based approach for finding the roots of two coupled nonlinear operators. Our focus is to characterize its finite-time performance in a Markov setting, which often…

Let $X_1,X_2,\ldots$ and $Y_1,Y_2,\ldots$ be two random sequences so that every random variable takes values in a finite set $\mathbb{A}$. We consider a global similarity score $L_n:=L(X_1,\ldots,X_n;Y_1,\ldots,Y_n)$ that measures the…