Related papers: Potentials of skip-free Markov chains
In this paper we consider stopping problems for continuous-time Markov chains under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. More precisely our aim is to maximize the certainty…
In this paper, we develop a general theory for the estimation of the transition probabilities of reversible Markov chains using the maximum entropy principle. A broad range of physical models can be studied within this approach. We use…
We consider an irreducible continuous time Markov chain on a finite state space and with time periodic jump rates and prove the joint large deviation principle for the empirical measure and flow and the joint large deviation principle for…
General characterizations of ergodic Markov chains have been developed in considerable detail. In this paper, we study the transience for discrete-time Markov chains on general state spaces, including the geometric transience and algebraic…
In this paper, we consider a general class of two-time-scale Markov chains whose transition rate matrix depends on a parameter $\lambda>0$. We assume that some transition rates of the Markov chain will tend to infinity as…
We introduce a general class of distances (metrics) between Markov chains, which are based on linear behaviour. This class encompasses distances given topologically (such as the total variation distance or trace distance) as well as by…
The paper studies the higher-order absolute differences taken from progressive terms of time-homogenous binary Markov chains. Two theorems presented are the limiting theorems for these differences, when their order $k$ converges to…
We develop the theory of the $W$ and $Z$ scale functions for right-continuous (upwards skip-free) discrete-time discrete-space random walks, along the lines of the analogue theory for spectrally negative L\'evy processes. Notably, we…
In the present paper, we investigate the relationship between hitting times and hitting probabilities in discrete-time imprecise Markov chains (IMCs). We define lower and upper hitting times and probabilities for IMCs whose set of…
Ergodicity is a fundamental issue for a stochastic process. In this paper, we refine results on ergodicity for a general type of Markov chain to a specific type or the $GI/G/1$-type Markov chain, which has many interesting and important…
Dealing with finite Markov chains in discrete time, the focus often lies on convergence behavior and one tries to make different copies of the chain meet as fast as possible and then stick together. There is, however, a very peculiar kind…
Higher-order Markov chains play a very important role in many fields, ranging from multilinear PageRank to financial modeling. In this paper, we propose three accelerated higher-order power methods for computing the limiting probability…
In this study, a new extension of the Markov Renewal theory is introduced by allowing time to evolve in multiple dimensions. The resulting chains are referred to as multi-time Markov Renewal chains and since this extension is new, the state…
The spatial symmetry property of truncated birth-death processes studied in Di Crescenzo [6] is extended to a wider family of continuous-time Markov chains. We show that it yields simple expressions for first-passage-time densities and…
We consider symmetric Markov chains on $\Bbb Z^d$ where we do {\bf not} assume that the conductance between two points must be zero if the points are far apart. Under a uniform second moment condition on the conductances, we obtain upper…
We obtain an upper escape rate function for a continuous time minimal symmetric Markov chain, defined on a locally finite weighted graph. This upper rate function is given in terms of volume growth with respect to an adapted path metric and…
We introduce a class of models for multidimensional control problems which we call skip-free Markov decision processes on trees. We describe and analyse an algorithm applicable to Markov decision processes of this type that are skip-free in…
In this paper we investigate the continuum limits of a class of Markov chains. The investigation of such limits is motivated by the desire to model very large networks. We show that under some conditions, a sequence of Markov chains…
Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…