Related papers: Skip-free Markov chains

Here we develop an effective approach to simplify two-time-scale Markov chains with infinite state spaces by removal of states with fast leaving rates, which improves the simplification method of finite Markov chains. We introduce the…

For downward skip-free continuous-time Markov chains on non-negative integers stopped at zero, existence of a quasi-stationary distribution is studied. The scale function for these processes is introduced and the boundary is classified by a…

We develop some sufficient conditions for the stochastic ordering between hitting times, in a fixed state, for two Markov chains. In particular, we focus attention on the so called \emph{skip-free} case. In the analysis of such a case, we…

A block Markov chain is a Markov chain whose state space can be partitioned into a finite number of clusters such that the transition probabilities only depend on the clusters. Block Markov chains thus serve as a model for Markov chains…

Two fundamental theorems by Spitzer/Erickson and Kesten/Maller on the fluctuation type (positive divergence, negative divergence or oscillation) of a real-valued random walk $(S_{n})_{n\ge 0}$ with iid increments $X_{1},X_{2},\ldots$ and…

A well-known theorem for an irreducible skip-free chain with absorbing state $d$, under some conditions, is that the hitting (absorbing) time of state $d$ starting from state 0 is distributed as the sum of $d$ independent geometric (or…

This thesis introduces a way to build Markov chains out of Hopf algebras. The transition matrix of a "Hopf-power Markov chain" is (the transpose of) the matrix of the coproduct-then-product operator on a combinatorial Hopf algebra with…

The purpose of this paper is to study the time average behavior of Markov chains with transition probabilities being kernels of completely continuous operators, and therefore to provide a sufficient condition for a class of Markov chains…

We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…

We present a new algorithm for the statistical model checking of Markov chains with respect to unbounded temporal properties, such as reachability and full linear temporal logic. The main idea is that we monitor each simulation run on the…

Perturbation theory for Markov chains addresses the question how small differences in the transitions of Markov chains are reflected in differences between their distributions. We prove powerful and flexible bounds on the distance of the…

We give an example of a transient reversible Markov chain that almost surely has only a finite number of cutpoints. We explain how this is relevant to a conjecture of Diaconis and Freedman and a question of Kaimanovich. We also answer…

In this paper, we are interested in investigating the perturbation bounds for the stationary distributions for discrete-time or continuous-time Markov chains on a countable state space. For discrete-time Markov chains, two new norm-wise…

Mahlmann and Schindelhauer (2005) defined a Markov chain which they called $k$-Flipper, and showed that it is irreducible on the set of all connected regular graphs of a given degree (at least 3). We study the 1-Flipper chain, which we call…

We consider a class of discrete time Markov chains with state space [0,1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then…

Continuous-time Markov chains on non-negative integers can be used for modeling biological systems, population dynamics, and queueing models. Qualitative behaviors of birth-and-death models, typical examples of such one-dimensional…

We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…

In any Markov chain with finite state space the distribution of transition records always belongs to the exponential family. This observation is used to prove a fluctuation theorem, and to show that the dynamical entropy of a stationary…

Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…

A discrete rate theory for general multi-ion channels is presented, in which the continuous dynamics of ion diffusion is reduced to transitions between Markovian discrete states. In an open channel, the ion permeation process involves three…