Related papers: Hitting times and interlacing eigenvalues: a stoch…
We study Markov processes where the "time" parameter is replaced by paths in a directed graph from an initial vertex to a terminal one. Along each directed path the process is Markov and has the same distribution as the one along any other…
We consider irreversible Markov chains on finite commutative rings randomly generated using both addition and multiplication. We restrict ourselves to the case where the addition is uniformly random and multiplication is arbitrary. We first…
The main subject of the study in this paper is the simultaneous renewal time for two time-inhomogeneous Markov chains which start with arbitrary initial distributions. By a simultaneous renewal we mean the first time of joint hitting the…
We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of…
For an ergodic Markov chain $\{X(t)\}$ on $\Bbb N$, with a stationary distribution $\pi$, let $T_n>0$ denote a hitting time for $[n]^c$, and let $X_n=X(T_n)$. Around 2005 Guy Louchard popularized a conjecture that, for $n\to \infty$, $T_n$…
We consider a strictly substochastic matrix or an stochastic matrix with absorbing states. By using quasi-stationary distributions one shows there is a canonical associated stationary Markov chain. Based upon $2-$stringing representation of…
We make use of the Open Quantum Random Walk setting due to S. Attal, F. Petruccione, C. Sabot and I. Sinayskiy [J. Stat. Phys. (2012) 147:832-852] in order to discuss hitting times and a quantum version of the Mean Hitting Time Formula from…
Perturbation theory for Markov chains addresses the question how small differences in the transitions of Markov chains are reflected in differences between their distributions. We prove powerful and flexible bounds on the distance of the…
This review paper provides an introduction of Markov chains and their convergence rates which is an important and interesting mathematical topic which also has important applications for very widely used Markov chain Monte Carlo (MCMC)…
Hidden semi-Markov Models (HSMM's) - while broadly in use - are restricted to a discrete and uniform time grid. They are thus not well suited to explain often irregularly spaced discrete event data from continuous-time phenomena. We show…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential…
In this paper, we analyze the dynamics of spreading processes taking place over time-varying networks. A common approach to model time-varying networks is via Markovian random graph processes. This modeling approach presents the following…
We define the hitting time for a Markov decision process (MDP). We do not use the hitting time of the Markov process induced by the MDP because the induced chain may not have a stationary distribution. Even it has a stationary distribution,…
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In…
We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the…
In this paper, we focus on the hitting times of a stochastic epidemic model presented by \cite{Gray}. Under the help of the auxiliary stopping times, we investigate the asymptotic limits of the hitting times by the variations of calculus…
Hitting times provide a fundamental measure of distance in random processes, quantifying the expected number of steps for a random walk starting at node $u$ to reach node $v$. They have broad applications across domains such as network…
Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the…
We study the problem of identity testing of markov chains. In this setting, we are given access to a single trajectory from a markov chain with unknown transition matrix $Q$ and the goal is to determine whether $Q = P$ for some known matrix…