Related papers: Numerical methods for the exit time of a piecewise…
In this paper we consider (upward skip-free) discrete-time and discrete-space Markov additive chains (MACs) and develop the theory for the so-called $\tilde{W}$ and $\tilde{Z}$ scale matrices. which are shown to play a vital role in the…
The purpose of this paper is to consider the exit-time problem for a finite-range Markov jump process, i.e, the distance the particle can jump is bounded independent of its location. Such jump diffusions are expedient models for anomalous…
Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with…
Stochastic mathematical models are essential tools for understanding and predicting complex phenomena. The purpose of this work is to study the exit times of a stochastic dynamical system-specifically, the mean exit time and the…
In order to approximate the exit time of a one-dimensional diffusion process, we propose an algorithm based on a random walk. Such an algorithm was already introduced in both the Brownian context and in the Ornstein-Uhlenbeck context. Here…
We study the time reversal of a general PDMP. The time reversed process is defined as $X_{(T-t)-}$, where $T$ is some given time and $X_t$ is a stationary PDMP. We obtain the parameters of the reversed process, like the jump intensity and…
Most exact algorithms for general partially observable Markov decision processes (POMDPs) use a form of dynamic programming in which a piecewise-linear and convex representation of one value function is transformed into another. We examine…
We present a numerical approximation technique for the analysis of continuous-time Markov chains that describe networks of biochemical reactions and play an important role in the stochastic modeling of biological systems. Our approach is…
We consider the problem of approximating the probability mass of the set of timed paths under a continuous-time Markov chain (CTMC) that are accepted by a deterministic timed automaton (DTA). As opposed to several existing works on this…
Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters…
We show a methodology for the computation of the probability of deadline miss for a periodic real-time task scheduled by a resource reservation algorithm. We propose a modelling technique for the system that reduces the computation of such…
Piecewise diffusion Markov processes (PDifMPs) form a versatile class of stochastic hybrid systems that combine continuous diffusion processes with discrete event-driven dynamics, enabling flexible modelling of complex real-world hybrid…
Variational formulas for the Laplace transform of the exit time from an open set of a Hunt process generated by a regular lower bounded semi-Dirichlet form are established. While for symmetric Markov processes, variational formulas are…
A biochemical network can be simulated by a set of ordinary differential equations (ODE) under well stirred reactor conditions, for large numbers of molecules, and frequent reactions. This is no longer a robust representation when some…
We establish $L^2$-exponential convergence rate for three popular piecewise deterministic Markov processes for sampling: the randomized Hamiltonian Monte Carlo method, the zigzag process, and the bouncy particle sampler. Our analysis is…
In decision modelling with time to event data, parametric models are often used to extrapolate the survivor function. One such model is the piecewise exponential model whereby the hazard function is partitioned into segments, with the…
Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to…
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
The paper deals with finite-state Markov decision processes (MDPs) with integer weights assigned to each state-action pair. New algorithms are presented to classify end components according to their limiting behavior with respect to the…