Related papers: Finiteness of hitting times under taboo

The continuous evolution of a wide variety of systems, including continuous-time Markov chains and linear hybrid automata, can be described in terms of linear differential equations. In this paper we study the decision problem of whether…

Consider the real Markov walk $S_n = X_1+ \dots+ X_n$ with increments $\left(X_n\right)_{n\geq 1}$ defined by a stochastic recursion starting at $X_0=x$. For a starting point $y>0$ denote by $\tau_y$ the exit time of the process $\left(…

Consider a discrete time, ergodic Markov chain with finite state space which is started from stationarity. Fill and Lyzinski (2014) showed that, in some cases, the hitting time for a given state may be represented as a sum of a geometric…

In the setting of non-reversible Markov chains on finite or countable state space, exact results on the distribution of the first hitting time to a given set $G$ are obtained. A new notion of "strong metastability time" is introduced to…

This paper studies continuous-time Markov decision processes under the risk-sensitive average cost criterion. The state space is a finite set, the action space is a Borel space, the cost and transition rates are bounded, and the…

We provide a framework for speeding up algorithms for time-bounded reachability analysis of continuous-time Markov decision processes. The principle is to find a small, but almost equivalent subsystem of the original system and only analyse…

Since many real-world problems arising in the fields of compiler optimisation, automated software engineering, formal proof systems, and so forth are equivalent to the Halting Problem--the most notorious undecidable problem--there is a…

In this short paper, we connect the procedure of constructing a totally inaccessible stopping time for a given process using the well-known Cox construction, dependent on an independent exponential random variable; with naturally occurring…

By choosing some special (random) initial data, we prove that with probability $1$, the stochastic shadow Gierer-Meinhardt system blows up pointwisely in finite time. We also give a (random) upper bound for the blowup time and some…

We consider the computational task of sampling a bit string $x$ from a distribution $\pi(x)=|\langle x|\psi\rangle|^2$, where $\psi$ is the unique ground state of a local Hamiltonian $H$. Our main result describes a direct link between the…

For any discrete target distribution, we exploit the connection between Markov chains and Stein's method via the generator approach and express the solution of Stein's equation in terms of expected hitting time. This yields new upper bounds…

Through a straightforward Bayesian approach we show that under some general conditions a maximum running time, namely the number of discrete steps performed by a computer program during its execution, can be defined such that the…

We consider the problem of approximating the reachability probabilities in Markov decision processes (MDP) with uncountable (continuous) state and action spaces. While there are algorithms that, for special classes of such MDP, provide a…

When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that…

We study discrete-time Markov chains on countably infinite state spaces, which are perturbed by rather general confining (i.e.\ growing at infinity) potentials. Using a discrete-time analogue of the classical Feynman--Kac formula, we obtain…

A branching process in a Markovian environment consists of an irreducible Markov chain on a set of "environments" together with an offspring distribution for each environment. At each time step the chain transitions to a new random…

Let (X_t, t>=0) be a Levy process started at 0, with Levy measure nu and T_x the first hitting time of level x>0: T_x:=inf{t>=0; X_t>x}. Let $F(theta, mu, rho,.) be the joint Laplace transform of (T_x, K_x, L_x): F(theta,mu,rho,x)…

We describe the solution of an optimal stopping problem for a stable L\'evy process killed at state-dependent rate, which can be seen as a model for bankruptcy. The killing rate is chosen in such a way that the killed process remains…

We study a probabilistic variant of binary session types that relate to a class of Finite-State Markov Chains. The probability annotations in session types enable the reasoning on the probability that a session terminates successfully, for…

We construct a simple example, surely known to Harry Kesten, of an R-transient Markov chain on a countable state space S with cemetery state delta. The transition matrix K on S is irreducible and strictly substochastic. We determine the…