Related papers: Almost triangular Markov chains on $\mathbb{N}$

Catastrophe Markov chain population models have received a lot of attention in the recent past. We herewith consider two special cases of such models involving total disasters, both in discrete and in continuous-time. Depending on the…

We aim at studying approximate null-controllability properties of a particular class of piecewise linear Markov processes (Markovian switch systems). The criteria are given in terms of algebraic invariance and are easily computable. We…

The birth-death process is a special type of continuous-time Markov chain with index set $\mathbb{N}$. Its resolvent matrix can be fully characterized by a set of parameters $(\gamma, \beta, \nu)$, where $\gamma$ and $\beta$ are…

Spectral methods have proven to be a highly effective tool in understanding the intrinsic geometry of a high-dimensional data set $\left\{x_i \right\}_{i=1}^{n} \subset \mathbb{R}^d$. The key ingredient is the construction of a Markov chain…

We consider Markov chains with random transition probabilities which, moreover, fluctuate randomly with time. We describe such a system by a product of stochastic matrices, $U(t)=M_t\cdots M_1$, with the factors $M_i$ drawn independently…

We obtain complementary recurrence and transience criteria for processes $X=(X_n)_{n \ge 0}$ with values in $\mathbb R^d_+$ fulfilling a non-linear equation $X_{n+1}=MX_n+g(X_n)+ \xi_{n+1}$. Here $M$ denotes a primitive matrix having…

In this paper, we develop an in-depth analysis of non-reversible Markov chains on denumerable state space from a similarity orbit perspective. In particular, we study the class of Markov chains whose transition kernel is in the similarity…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

Let $X_n, n \ge 0$ be a Markov chain with finite state space $M$. If $x,y \in M$ such that $x$ is transient we have $P^y(X_n = x) \to 0$ for $n \to \infty$, and under mild aperiodicity conditions this convergence is monotone in that for…

The $L^p$ maximal inequalities for martingales are one of the classical results in probability theory. Here we establish the sharp moderate maximal inequalities for upward skip-free Markov chains, which include the $L^p$ maximal…

In the first part of this thesis, we study a Markov chain on $\mathbb{R}_+ \times S$, where $\mathbb{R}_+$ is the non-negative real numbers and $S$ is a finite set, in which when the $\mathbb{R}_+$-coordinate is large, the $S$-coordinate of…

We consider the time dependent probability distribution of a coarse grained observable Y whose evolution is governed by a discrete time map. If the map is mixing, the time dependent one-step transition probabilities converge in the long…

In this article almost semi-continuous processes with stationary independent increments on a finite irreducible Markov chain are considered. For these processes the components of matrix factorization identity are concretely defined. On the…

We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing…

In this paper we consider large state space continuous time Markov chains (MCs) arising in the field of systems biology. For density dependent families of MCs that represent the interaction of large groups of identical objects, Kurtz has…

An up-down chain is a Markov chain in which each transition is a two-step process that moves up to a larger object and then back down to an object of the original size. The first goal of this paper is to present a general framework for…

Consider a finite irreducible Markov chain with invariant probability $\pi$. Define its inverse communication speed as the expectation to go from x to y, when x, y are sampled independently according to $\pi$. In the discrete time setting…

For i.i.d. random vectors $(M_{1},Q_{1}),(M_{2},Q_{2}),\ldots$ such that $M>0$ a.s., $Q\geq 0$ a.s. and $\mathbb{P}(Q=0)<1$, the random difference equation $X_{n}=M_{n}X_{n-1}+Q_{n}$, $n=1,2,\ldots$, is studied in the critical case when the…

For downward skip-free continuous-time Markov chains on non-negative integers stopped at zero, existence of a quasi-stationary distribution is studied. The scale function for these processes is introduced and the boundary is classified by a…

This paper is concerned with ergodic properties of inhomogeneous Markov processes. Since the transition probabilities depend on initial times, the existing methods to obtain invariant measures for homogeneous Markov processes are not…