Related papers: Waiting times and stopping probabilities for patte…

The paper is largely of a review nature. It considers two main methods used to study stability and obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable…

Consider a Markov chain with finite state $\{0, 1, ..., d\}$. We give the generation functions (or Laplace transforms) of absorbing (passage) time in the following two situations : (1) the absorbing time of state $d$ when the chain starts…

Consider a sequence of continuous-time irreducible reversible Markov chains and a sequence of initial distributions, $\mu_n$. The sequence is said to exhibit $\mu_n$-cutoff if the convergence to stationarity in total variation distance is…

In this research the technology of complex Markov chains is applied to predict financial time series. The main distinction of complex or high-order Markov Chains and simple first-order ones is the existing of aftereffect or memory. The…

We consider a Markov chain $(M_{n})_{n\ge 0}$ on the set $\mathbb{N}_{0}$ of nonnegative integers which is eventually decreasing, i.e. $\mathbb{P}\{M_{n+1}<M_{n}|M_{n}\ge a\}=1$ for some $a\in\mathbb{N}$ and all $n\ge 0$. We are interested…

Consider the following Markov chain on permutations of length $n$. At each time step we choose a random position. If the letter at that position is smaller than the letter immediately to the left (cyclically) then these letters swap…

We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these…

In this paper, we consider the Markov-Chain Monte Carlo (MCMC) approach for random sampling of combinatorial objects. The running time of such an algorithm depends on the total mixing time of the underlying Markov chain and is unknown in…

In this paper, we develop a more general framework of block-structured Markov processes in the queueing study of blockchain systems, which can provide analysis both for the stationary performance measures and for the sojourn times of any…

We present a novel method for computing reachability probabilities of parametric discrete-time Markov chains whose transition probabilities are fractions of polynomials over a set of parameters. Our algorithm is based on two key…

We suggest a non-asymptotic matrix perturbation-theoretic approach to get sharp bounds on the expected meeting time of random walks on large (possibly random) graphs. We provide a formula for the expected meeting time in terms of the…

We consider the problem of bounding mean first passage times for a class of continuous-time Markov chains that captures stochastic interactions between groups of identical agents. The quantitative analysis of such probabilistic population…

Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…

In this paper we consider stopping problems for continuous-time Markov chains under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. More precisely our aim is to maximize the certainty…

Let $(X_n \colon n\in\Z)$ be a two-sided recurrent Markov chain with fixed initial state $X_0$ and let $\nu$ be a probability measure on its state space. We give a necessary and sufficient criterion for the existence of a non-randomized…

We study continuous-time Markov chains on the non-negative integers under mild regularity conditions (in particular, the set of jump vectors is finite and both forward and backward jumps are possible). Based on the so-called flux balance…

Given a sequence $(\mathfrak{X}_i, \mathscr{K}_i)_{i=1}^\infty$ of Markov chains, the cut-off phenomenon describes a period of transition to stationarity which is asymptotically lower order than the mixing time. We study mixing times and…

This article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30,197-207, (1986)…

Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters…

The aim of this note is to construct a probability measure on the space of trajectories in a continuous time Markov chain having a finite state diagram, or more generally which admits a global bound on its degree and rates. Our approach is…