Related papers: Graph Transformation Method for Calculating Waitin…
Consider a system evolving according to an absorbing discrete-time Markov chain with known transition matrix. The state of the system is observed at two points in time, separated by an unknown number of generations. We are interested in…
We determine the asymptotic speed of the first-passage percolation process on some ladder-like graphs (or width-2 stretches) when the times associated with different edges are independent and exponentially distributed but not necessarily…
In source routing, a complete path is chosen for a packet to travel from source to destination. While computing the time to traverse such a path may be straightforward in a fixed, static graph, doing so becomes much more challenging in…
Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a…
We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…
Large continuous-time Markov chains with exponentially small transition rates arise in modeling complex systems in physics, chemistry and biology. We propose a constructive graph-algorithmic approach to determine the sequence of critical…
We make use of matrix representations of completely positive maps in order to study open quantum dynamics on graphs, with emphasis on quantum walks and the associated trajectories obtained via a monitoring of the position. We discuss the…
We propose a direct numerical method to calculate the statistics of the number of transitions in stochastic processes, without having to resort to Monte Carlo calculations. The method is based on a generating function method, and arbitrary…
There are known expressions to calculate the moments of the first passage time in Markov chains. Nevertheless, it is commonly forgotten that in most applications the parameters of the Markov chain are constructed using estimates based upon…
In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent…
The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…
We compute the stationary distribution of a continuous-time Markov chain which is constructed by gluing together two finite, irreducible Markov chains by identifying a pair of states of one chain with a pair of states of the other and…
We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…
Flowgraph models provide an alternative approach in modeling a multi-state stochastic process. One of the most widely used stochastic processes that have many real-world applications especially in actuarial models is the Markov jump process…
In this paper we address the problem of the calculation of the mean first passage time (MFPT) on generic graphs. We focus in particular on the mean first passage time on a node 's' for a random walker starting from a generic, unknown, node…
Probability generating functions for first passage times of Markov chains are found using the method of collective marks. A system of equations is found which can be used to obtain moments of the first passage times.
Markov switching models are a popular family of models that introduces time-variation in the parameters in the form of their state- or regime-specific values. Importantly, this time-variation is governed by a discrete-valued latent…
We study the accurate and efficient computation of the expected number of times each state is visited in discrete- and continuous-time Markov chains. To obtain sound accuracy guarantees efficiently, we lift interval iteration and…