Related papers: Almost triangular Markov chains on $\mathbb{N}$

In this paper, we consider a general class of two-time-scale Markov chains whose transition rate matrix depends on a parameter $\lambda>0$. We assume that some transition rates of the Markov chain will tend to infinity as…

We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential…

In this paper, we study a biased version of the nearest-neighbor transposition Markov chain on the set of permutations where neighboring elements $i$ and $j$ are placed in order $(i,j)$ with probability $p_{i,j}$. Our goal is to identify…

Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…

This paper presents a new matrix-infinite-product-form (MIP-form) solution for the stationary distribution in upper block-Hessenberg Markov chains (UBH-MCs). The existing MIP-form solution (Masuyama, Queueing Syst., Vol. 92, 2019, pp.…

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…

We study the limit behaviour of upper and lower bounds on expected time averages in imprecise Markov chains; a generalised type of Markov chain where the local dynamics, traditionally characterised by transition probabilities, are now…

We study the convergence rate to stationarity for a class of exchangeable partition-valued Markov chains called cut-and-paste chains. The law governing the transitions of a cut-and-paste chain are determined by products of i.i.d. stochastic…

In this paper we propose a new method for approximating the nonstationary moment dynamics of one dimensional Markovian birth-death processes. By expanding the transition probabilities of the Markov process in terms of Poisson-Charlier…

A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…

The recently found hypergeometric multiple orthogonal polynomials on the step-line by Lima and Loureiro are shown to be random walk polynomials. It is proven that the corresponding Jacobi matrix and its transpose, which are nonnegative…

We develop a general theory of "almost Hadamard matrices". These are by definition the matrices $H\in M_N(\mathbb R)$ having the property that $U=H/\sqrt{N}$ is orthogonal, and is a local maximum of the 1-norm on O(N). Our study includes a…

Many examples of exactly solvable birth and death processes, a typical stationary Markov chain, are presented together with the explicit expressions of the transition probabilities. They are derived by similarity transforming exactly…

We consider a family of skew-products of the form $(Tx, g_x(t)) : X \times \mathbb{R} \to X \times \mathbb{R}$ where $T$ is a continuous expanding Markov map and $g_x : \mathbb{R} \to \mathbb{R}$ is a family of homeomorphisms of…

This book is an introduction to quantum Markov chains and explains how this concept is connected to the question of how well a lost quantum mechanical system can be recovered from a correlated subsystem. To achieve this goal, we strengthen…

The invariant measure is a fundamental object in the theory of Markov processes. In finite dimensions a Markov process is defined by transition rates of the corresponding stochastic matrix. The Markov tree theorem provides an explicit…

We study a large class of reversible Markov chains with discrete state space and transition matrix $P_N$. We define the notion of a set of {\it metastable points} as a subset of the state space $\G_N$ such that (i) this set is reached from…

The class of stochastic matrices that have a stochastic $c$-th root for infinitely many natural numbers $c$ is introduced and studied. Such matrices are called arbitrarily finely divisible, and generalise the class of infinitely divisible…

The upper extremes of a Markov chain with regulary varying stationary marginal distribution are known to exhibit under general conditions a multiplicative random walk structure called the tail chain. More generally, if the Markov chain is…

We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…