Related papers: Tight Markov chains and random compositions

In this article we provide new applications for exponential approximation using the framework of Pek\"oz and R\"ollin (in press), which is based on Stein's method. We give error bounds for the nearly critical Galton-Watson process…

We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…

By proving a local limit theorem for higher-order transitions, we determine the time required for necklace chains to be close to stationarity. Because necklace chains, built by arranging identical smaller chains around a directed cycle, are…

Proving a 2009 conjecture of Itai Benjamini, we show: For any C there is an $\varepsilon>0$ such that for any simple graph $G$ on $V$ of size $n$, and $X_0,\ldots$ an ordinary random walk on $G$, $P(\{X_0,\dots, X_{Cn}\}= V) <…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

We study a large class of reversible Markov chains with discrete state space and transition matrix $P_N$. We define the notion of a set of {\it metastable points} as a subset of the state space $\G_N$ such that (i) this set is reached from…

We examine two analytical characterisation of the metastable behavior of a Markov chain. The first one expressed in terms of its transition probabilities, and the second one in terms of its large deviations rate functional. Consider a…

We study stochastic approximation procedures for approximately solving a $d$-dimensional linear fixed point equation based on observing a trajectory of length $n$ from an ergodic Markov chain. We first exhibit a non-asymptotic bound of the…

We observe a length-$n$ sample generated by an unknown,stationary ergodic Markov process (\emph{model}) over a finite alphabet $\mathcal{A}$. Given any string $\bf{w}$ of symbols from $\mathcal{A}$ we want estimates of the conditional…

We consider again the fast-slow motions setups in the continuous time $\frac {dX_N(t)}{dt}=N^{1/2} \sig(X_N(t))(\xi(tN))+b(X_N(t)),\, t\in [0,T]$ and the discrete time $X_N((n+1)/N)=X_N(n/N)+N^{-1/2}\sig(X_N(n/N))\xi(n)+N^{-1}b(X_N(n/N)),\,…

We provide a unified framework to compute the stationary distribution of any finite irreducible Markov chain or equivalently of any irreducible random walk on a finite semigroup $S$. Our methods use geometric finite semigroup theory via the…

A Markov chain is geometrically ergodic if it converges to its in- variant distribution at a geometric rate in total variation norm. We study geo- metric ergodicity of deterministic and random scan versions of the two-variable Gibbs…

In this paper, we study quasi-stationary distributions of nonlinearly perturbed semi-Markov processes in discrete time. This type of distributions is of interest for the analysis of stochastic systems which have finite lifetimes, but are…

We consider a family of continuous time symmetric random walks indexed by $k\in \mathbb{N}$, $\{X_k(t),\,t\geq 0\}$. For each $k\in \mathbb{N}$ the matching random walk take values in the finite set of states…

We establish novel and general high-dimensional concentration inequalities and Berry-Esseen bounds for vector-valued martingales induced by Markov chains. We apply these results to analyze the performance of the Temporal Difference (TD)…

We analyse the $\ell^2(\pi)$-convergence rate of irreducible and aperiodic Markov chains with $N$-band transition probability matrix $P$ and with invariant distribution $\pi$. This analysis is heavily based on: first the study of the…

We consider local alignments without gaps of two independent Markov chains from a finite alphabet, and we derive sufficient conditions for the number of essentially different local alignments with a score exceeding a high threshold to be…

We study distribution of orbits sampled at polynomial times for uniquely ergodic topological dynamical systems $(X, T)$. First, we prove that if there exists an increasing sequence $(q_n)$ for which the rigidity condition \[…

This note presents conjectures on polynomial/algebraic/sub-exponential convergence of transition probabilities for $\lambda$-null recurrent and $\lambda$-transient Markov chains in continuous time. The only known positive examples are in…

Let $(X, Y) = (X_n, Y_n)_{n \geq 1}$ be the output process generated by a hidden chain $Z = (Z_n)_{n \geq 1}$, where $Z$ is a finite state, aperiodic, time homogeneous, and irreducible Markov chain. Let $LC_n$ be the length of the longest…