Related papers: Relation between Hitting Times and Probabilities f…
Hidden Markov Chains (HMCs) are commonly used mathematical models of probabilistic systems. They are employed in various fields such as speech recognition, signal processing, and biological sequence analysis. We consider the problem of…
We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…
Hitting times for discrete quantum walks on graphs give an average time before the walk reaches an ending condition. To be analogous to the hitting time for a classical walk, the quantum hitting time must involve repeated measurements as…
In recent years probabilistic model checking has become an important area of research because of the diffusion of computational systems of stochastic nature. Despite its great success, standard probabilistic model checking suffers the…
In this paper, we present a novel iterative Monte Carlo method for approximating the stationary probability of a single state of a positive recurrent Markov chain. We utilize the characterization that the stationary probability of a state…
In this paper, we show how to use stochastic approximation to compute hitting time of a stochastic process, based on the study of the time for a fluid approximation of this process to be at distance 1/N of its fixed point. This approach is…
The standard approach to realize a quantum repeater relies upon probabilistic but heralded entangled state manipulations and the storage of quantum states while waiting for successful events. In the literature on this class of repeaters,…
Starting from an n-by-n matrix of zeros, choose uniformly random zero entries and change them to ones, one-at-a-time, until the matrix becomes invertible. We show that with probability tending to one as n tends to infinity, this occurs at…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the…
The paper addresses the problem of computing maximal conditional expected accumulated rewards until reaching a target state (briefly called maximal conditional expectations) in finite-state Markov decision processes where the condition is…
We introduce a Markov chain model of concurrent quantum programs. This model is a quantum generalization of Hart, Sharir and Pnueli's probabilistic concurrent programs. Some characterizations of the reachable space, uniformly repeatedly…
This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the…
We investigate performance of approximations put forth in \citeNP{[Malinovskii 2017a]} and \citeNP{[Malinovskii 2017b]} for the distribution of the time of first level $u$ crossing by the random process $\homV{s}-cs$, $s>0$, where…
Let $\hat m_{ij}$ be the hitting (mean first passage) time from state $i$ to state $j$ in an $n$-state ergodic homogeneous Markov chain with transition matrix $T$. Let $\Gamma$ be the weighted digraph whose vertex set coincides with the set…
We explore the concept of a consistent exchangeable survival process - a joint distribution of survival times in which the risk set evolves as a continuous-time Markov process with homogeneous transition rates. We show a correspondence with…
We study the hitting times of Markov processes to target set $G$, starting from a reference configuration $x_0$ or its basin of attraction. The configuration $x_0$ can correspond to the bottom of a (meta)stable well, while the target $G$…
In the present paper we show that for any given digraph $\mathbb{G} =([n], \vec{E})$, i.e. an oriented graph without self-loops and 2-cycles, one can construct a 1-dependent Markov chain and $n$ identically distributed hitting times $T_1,…
We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of…
Continuous-time quantum walks have proven to be an extremely useful framework for the design of several quantum algorithms. Often, the running time of quantum algorithms in this framework is characterized by the quantum hitting time: the…