Related papers: Skip-free Markov chains

A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…

In this work, we study a natural nonparametric estimator of the transition probability matrices of a finite controlled Markov chain. We consider an offline setting with a fixed dataset, collected using a so-called logging policy. We develop…

We consider the two-dimensional simple random walk conditioned on never hitting the origin. This process is a Markov chain, namely it is the Doob $h$-transform of the simple random walk with respect to the potential kernel. It is known to…

The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…

For a stochastically monotone Markov chain taking values in a Polish space, we present a number of conditions for existence and for uniqueness of its stationary regime, as well as for closeness of its transient trajectories. In particular,…

We consider a class of optimal control problems, with finite or infinite horizon, for a continuous-time Markov chain with finite state space. In this case, the control process affects the transition rates. We suppose that the controlled…

Random walks on simple graphs in connection with electrical resistor networks lead to the definition of Markov chains with transition probability matrix in terms of electrical conductances. We extend this definition to an effective…

We introduce a new framework that yields spectral bounds on norms of functions of transition maps for finite, homogeneous Markov chains. The techniques employed work for bounded semigroups, in particular for classical as well as for quantum…

We prove a large deviation principle on path space for a class of discrete time Markov processes whose state space is the intersection of a regular domain $\L\subset \R^d$ with some lattice of spacing $\e$. Transitions from $x$ to $y$ are…

We extend Hoeffding's lemma to general-state-space and not necessarily reversible Markov chains. Let $\{X_i\}_{i \ge 1}$ be a stationary Markov chain with invariant measure $\pi$ and absolute spectral gap $1-\lambda$, where $\lambda$ is…

We consider the problem of identity testing of Markov chain transition matrices based on a single trajectory of observations under the distance notion introduced by Daskalakis et al. [2018a] and further analyzed by Cherapanamjeri and…

Multiple works regarding convergence analysis of Markov chains have led to spectral gap decomposition formulas of the form \[ \mathrm{Gap}(S) \geq c_0 \left[\inf_z \mathrm{Gap}(Q_z)\right] \mathrm{Gap}(\bar{S}), \] where $c_0$ is a…

Consider longitudinal networks whose edges turn on and off according to a discrete-time Markov chain with exponential-family transition probabilities. We characterize when their joint distributions are also exponential families with the…

A stable-like Markov chain is a time-homogeneous Markov chain on the real line with the transition kernel $p(x,dy)=f_x(y-x)dy$, where the density functions $f_x(y)$, for large $|y|$, have a power-law decay with exponent $\alpha(x)+1$, where…

The aim of this note is to present an elementary proof of a variation of Harris' ergodic theorem of Markov chains. This theorem, dating back to the fifties essentially states that a Markov chain is uniquely ergodic if it admits a ``small''…

We develop a criterion for transience for a general model of branching Markov chains. In the case of multi-dimensional branching random walk in random environment (BRWRE) this criterion becomes explicit. In particular, we show that…

Expanding upon the rich history of algebraic techniques in probability, we show the existence of and construct a Markov chain using the Hopf square map on a quantum group that is both non-commutative and non-cocommutative. This extends the…

We consider Markov chains that obey the following general non-linear state space model: $\Phi_{k+1} = F(\Phi_k, \alpha(\Phi_k, U_{k+1}))$ where the function $F$ is $C^1$ while $\alpha$ is typically discontinuous and $\{U_k: k \in…

We study convergence to equilibrium for a large class of Markov chains in random environment. The chains are sparse in the sense that in every row of the transition matrix $P$ the mass is essentially concentrated on few entries. Moreover,…

We consider irreversible Markov chains on finite commutative rings randomly generated using both addition and multiplication. We restrict ourselves to the case where the addition is uniformly random and multiplication is arbitrary. We first…