Related papers: Skip-free Markov chains

The aim of this paper is to study the fluctuations of a general class of supercritical branching Markov processes with non-local branching mechanisms. We show the existence of three regimes according to the size of the spectral gap…

In this paper we develop a statistical estimation technique to recover the transition kernel $P$ of a Markov chain $X=(X_m)_{m \in \mathbb N}$ in presence of censored data. We consider the situation where only a sub-sequence of $X$ is…

We introduce the avoidance Markov metrics and theories which provide more flexibility in the design of random walk and impose new conditions on the walk to avoid (or transit) a specific node (or a set of nodes) before the stopping criteria.…

We consider a class of continuous time Markov chains on $\Z^d$. These chains are the discrete space analogue of Markov processes with jumps. Under some conditions, we show that harmonic functions associated with these Markov chains are…

We introduce a class of models for multidimensional control problems which we call skip-free Markov decision processes on trees. We describe and analyse an algorithm applicable to Markov decision processes of this type that are skip-free in…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…

Random walks (or Markov chains) are models extensively used in theoretical computer science. Several tools, including analysis of quantities such as hitting and mixing times, are helpful for devising randomized algorithms. A notable example…

We consider the recurrence and transience problem for a time-homogeneous Markov chain on the real line with transition kernel $p(x,\mathrm{d}y)=f_x(y-x)\,\mathrm{d}y$, where the density functions $f_x(y)$, for large $|y|$, have a power-law…

We consider an elementary model for self-organised criticality, the activated random walk on the complete graph. We introduce a discrete time Markov chain as follows. At each time step, we add an active particle at a random vertex and let…

Let $\{\boldsymbol{X}_n\}$ be a discrete-time $d$-dimensional process on $\mathbb{Z}_+^d$ with a supplemental (background) process $\{J_n\}$ on a finite set and assume the joint process $\{\boldsymbol{Y}_n\}=\{(\boldsymbol{X}_n,J_n)\}$ to…

We consider Markov chains with random transition probabilities which, moreover, fluctuate randomly with time. We describe such a system by a product of stochastic matrices, $U(t)=M_t\cdots M_1$, with the factors $M_i$ drawn independently…

The $L^p$ maximal inequalities for martingales are one of the classical results in probability theory. Here we establish the sharp moderate maximal inequalities for upward skip-free Markov chains, which include the $L^p$ maximal…

Given a non-negative Jacobi matrix describing higher order recurrence relations for multiple orthogonal polynomials of type~II and corresponding linear forms of type I, a general strategy for constructing a pair of stochastic matrices, dual…

Let us consider a homogeneous Markov chain with discrete time and with a finite set of states $E_0,\ldots,E_n$ such that the state $E_0$ is absorbing, states $E_1,\ldots,E_n$ are nonrecurrent. The goal of this work is to study frequencies…

We present a recurrence-transience classification for discrete-time Markov chains on manifolds with negative curvature. Our classification depends only on geometric quantities associated to the increments of the chain, defined via the…

Nonlinear Markov chains with finite state space have been introduced in Kolokoltsov (2010). The characteristic property of these processes is that the transition probabilities do not only depend on the state, but also on the distribution of…

Let M be a noncompact metric space in which every closed ball is compact, and let G be a semigroup of Lipschitz mappings of M. Denote by (Y_n)_{n\geq1} a sequence of independent G-valued, identically distributed random variables (r.v.'s),…

This paper studies Markov chains on the symmetric group $S_n$ where the transition probabilities are given by the Ewens distribution with parameter $\theta>1$. The eigenvalues are identified to be proportional to the content polynomials of…

We study Markov chains on $\mathbb Z^m$, $m\geq 2$, that behave like a standard symmetric random walk outside of the hyperplane (membrane) $H=\{0\}\times \mathbb Z^{m-1}$. The transition probabilities on the membrane $H$ are periodic and…