Related papers: Hitting Times and Probabilities for Imprecise Mark…

We investigate the hitting times of random walks on graphs, where a hitting time is defined as the number of steps required for a random walker to move from one node to another. While much of the existing literature focuses on calculating…

Consider a discrete time, ergodic Markov chain with finite state space which is started from stationarity. Fill and Lyzinski (2014) showed that, in some cases, the hitting time for a given state may be represented as a sum of a geometric…

The paper is largely of a review nature. It considers two main methods used to study stability and obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable…

This study introduces a novel approach for learning mixtures of Markov chains, a critical process applicable to various fields, including healthcare and the analysis of web users. Existing research has identified a clear divide in…

It is well known that the distributions of hitting times in Markov chains are quite irregular, unless the limit as time tends to infinity is considered. We show that nevertheless for a typical finite irreducible Markov chain and for…

We consider irreducible reversible discrete time Markov chains on a finite state space. Mixing times and hitting times are fundamental parameters of the chain. We relate them by showing that the mixing time of the lazy chain is equivalent…

Interval Markov chains extend classical Markov chains with the possibility to describe transition probabilities using intervals, rather than exact values. While the standard formulation of interval Markov chains features closed intervals,…

Random walks (or Markov chains) are models extensively used in theoretical computer science. Several tools, including analysis of quantities such as hitting and mixing times, are helpful for devising randomized algorithms. A notable example…

Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…

We develop some sufficient conditions for the stochastic ordering between hitting times, in a fixed state, for two Markov chains. In particular, we focus attention on the so called \emph{skip-free} case. In the analysis of such a case, we…

The distribution of the "mixing time" or the "time to stationarity" in a discrete time irreducible Markov chain, starting in state i, can be defined as the number of trials to reach a state sampled from the stationary distribution of the…

We consider the problem of estimating the transition rate matrix of a continuous-time Markov chain from a finite-duration realisation of this process. We approach this problem in an imprecise probabilistic framework, using a set of prior…

We study the limit behaviour of upper and lower bounds on expected time averages in imprecise Markov chains; a generalised type of Markov chain where the local dynamics, traditionally characterised by transition probabilities, are now…

The hitting and mixing times are two fundamental quantities associated with Markov chains. In Peres and Sousi[PS2015] and Oliveira[Oli2012], the authors show that the mixing times and "worst-case" hitting times of reversible Markov chains…

In recent years probabilistic model checking has become an important area of research because of the diffusion of computational systems of stochastic nature. Despite its great success, standard probabilistic model checking suffers the…

In the paper, we study a new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains based on the Markov-Dobrushin condition. This result generalizes the convergence estimates for any positive number of transition…

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the…

The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the…

This paper focuses on a class of continuous-time controlled Markov chains with time-inconsistent and distribution-dependent cost functional (in some appropriate sense). A new definition of time-inconsistent distribution-dependent…

Markov chain analysis is a key technique in formal verification. A practical obstacle is that all probabilities in Markov models need to be known. However, system quantities such as failure rates or packet loss ratios, etc. are often not --…