Related papers: Convergence of a Second Order Markov Chain
The switch Markov chain has been extensively studied as the most natural Markov Chain Monte Carlo approach for sampling graphs with prescribed degree sequences. We use comparison arguments with other, less natural but simpler to analyze,…
The class of nonlinear Markov processes is characterized by the dependence of the current state of the process on its current distribution in addition to the dependence on the previous state. Due to this feature, these processes are…
Extensions of Kemeny's constant, as derived for irreducible finite Markov chains in discrete time, to Markov renewal processes and Markov chains in continuous time are discussed. Three alternative Kemeny's functions and their variants are…
Dealing with finite Markov chains in discrete time, the focus often lies on convergence behavior and one tries to make different copies of the chain meet as fast as possible and then stick together. There is, however, a very peculiar kind…
Consider continuous-time linear switched systems on R^n associated with compact convex sets of matrices. When the system is irreducible and the largest Lyapunov exponent is equal to zero, there always exists a Barabanov norm (i.e. a norm…
Large deviation theory is a branch of probability theory that is devoted to a study of the "rate" at which empirical estimates of various quantities converge to their true values. The object of study in this paper is the rate at which…
We consider continuous-time Markov chains which display a family of wells at the same depth. We provide sufficient conditions which entail the convergence of the finite-dimensional distributions of the order parameter to the ones of a…
Stochastic optimization methods such as mirror descent have wide applications due to low computational cost. Those methods have been well studied under assumption of the independent and identical distribution, and usually achieve sublinear…
We consider sequences $(X_t^N)_{t\geq0}$ of Markov processes in two dimensions whose fluid limit is a stable solution of an ordinary differential equation of the form $\dot{x}_t=b(x_t)$, where $b(x)={\pmatrix{-\mu 0 0 \lambda}}x+\tau(x)$…
Parallel tempering (PT) methods are a popular class of Markov chain Monte Carlo schemes used to sample complex high-dimensional probability distributions. They rely on a collection of $N$ interacting auxiliary chains targeting tempered…
The embedding problem for Markov chains is a famous problem in probability theory and only partial results are available up till now. In this paper, we propose a variant of the embedding problem called the reversible embedding problem which…
We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…
A theory of symbolic dynamic systems with long-range correlations based on the consideration of the binary N-step Markov chains developed earlier in Phys. Rev. Lett. 90, 110601 (2003) is generalized to the biased case (non equal numbers of…
In this work we introduce a new and richer class of finite order Markov chain models and address the following model selection problem: find the Markov model with the minimal set of parameters (minimal Markov model) which is necessary to…
We consider a form of state-dependent drift condition for a general Markov chain, whereby the chain subsampled at some deterministic time satisfies a geometric Foster-Lyapunov condition. We present sufficient criteria for such a drift…
We show that the convergence of finite state space Markov chains to stationarity can often be considerably speeded up by alternating every step of the chain with a deterministic move. Under fairly general conditions, we show that not only…
Recently it has been conjectured that the ground-state of a Markovian Hamiltonian, with one boundary operator, acting in a link pattern space is related to vertically and horizontally symmetric alternating-sign matrices (equivalently…
In this paper, we consider general Markov chains (MC), specified by the transition probability (kernel) $ P (x, E) $, finitely additive in the second argument. Such MC are studied within the framework of the functional operator treatment.…
We present here two standalone results from a forthcoming work on the analysis of Markov chains using the representation theory of $S_n$. First, we give explicit formulas for the decompositions of tensor powers of the defining and standard…
Aldous-Broder algorithm is a famous algorithm used to sample a uniform spanning tree of any finite connected graph $G$, but it is more general: given an irreducible and reversible Markov chain $M$ on $G$ started at $r$, the tree rooted at…