Related papers: Ergodic Theorems for discrete Markov chains

Consider a discrete time, ergodic Markov chain with finite state space which is started from stationarity. Fill and Lyzinski (2014) showed that, in some cases, the hitting time for a given state may be represented as a sum of a geometric…

We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…

A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…

A finite ergodic Markov chain is said to exhibit cutoff if its distance to stationarity remains close to 1 over a certain number of iterations and then abruptly drops to near 0 on a much shorter time scale. Discovered in the context of card…

Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential…

A classical random walk $(S_t, t\in\mathbb{N})$ is defined by $S_t:=\displaystyle\sum_{n=0}^t X_n$, where $(X_n)$ are i.i.d. When the increments $(X_n)_{n\in\mathbb{N}}$ are a one-order Markov chain, a short memory is introduced in the…

We consider the problem of finding the optimal upper bound for the tail probability of a sum of $k$ nonnegative, independent and identically distributed random variables with given mean $x$. For $k=1$ the answer is given by Markov's…

Consider a Markov process $\{\Phi(t) : t\geq 0\}$ evolving on a Polish space ${\sf X}$. A version of the $f$-Norm Ergodic Theorem is obtained: Suppose that the process is $\psi$-irreducible and aperiodic. For a given function $f\colon{\sf…

Let $\{X_n\}_{n\ge0}$ be a $V$-geometrically ergodic Markov chain. Given some real-valued functional $F$, define $M_n(\alpha):=n^{-1}\sum_{k=1}^nF(\alpha,X_{k-1},X_k)$, $\alpha\in\mathcal{A}\subset \mathbb {R}$. Consider an $M$ estimator…

The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…

The limiting probability distribution is one of the key characteristics of a Markov chain since it shows its long-term behavior. In this paper, for a higher order Markov chain, we establish some properties related to its exact limiting…

Let p_n denote the persistence probability that the first n iterated partial sums of integrable, zero-mean, i.i.d. random variables X_k, are negative. We show that p_n is bounded above up to universal constant by the square root of the…

In this paper, we establish moment and Bernstein-type inequalities for additive functionals of geometrically ergodic Markov chains. These inequalities extend the corresponding inequalities for independent random variables. Our conditions…

Many classical randomized algorithms (e.g., approximation algorithms for #P-complete problems) utilize the following random walk algorithm for {\em almost uniform sampling} from a state space $S$ of cardinality $N$: run a symmetric ergodic…

We consider refined versions of Markov chains related to juggling introduced by Warrington. We further generalize the construction to juggling with arbitrary heights as well as infinitely many balls, which are expressed more succinctly in…

Establishing central limit theorems (CLTs) for ergodic averages of Markov chains is a fundamental problem in probability and its applications. Since the seminal work~\cite{MR834478}, a vast literature has emerged on the sufficient…

We study branching Markov chains on a countable state space (space of types) $\mathscr{X}$, with the focus on the qualitative aspects of the limit behaviour of the evolving empirical population distributions. No conditions are imposed on…

The classic arcsine law for the number $N_{n}^{>}:=n^{-1}\sum_{k=1}^{n}\mathbf{1}_{\{S_{k}>0\}}$ of positive terms, as $n\to\infty$, in an ordinary random walk $(S_{n})_{n\ge 0}$ is extended to the case when this random walk is governed by…

We consider a time-average estimator $f_{k}$ of a functional of a Markov chain. Under a coupling assumption, we show that the expectation of $f_{k}$ has a limit $\mu$ as the number of time-steps goes to infinity. We describe a modification…

We study convergence to equilibrium for a large class of Markov chains in random environment. The chains are sparse in the sense that in every row of the transition matrix $P$ the mass is essentially concentrated on few entries. Moreover,…