Related papers: Timed Comparisons of Semi-Markov Processes

In most sampling algorithms, including Hamiltonian Monte Carlo, transition rates between states correspond to the probability of making a transition in a single time step, and are constrained to be less than or equal to 1. We derive a…

This paper introduces the concept of random context representations for the transition probabilities of a finite-alphabet stochastic process. Processes with these representations generalize context tree processes (a.k.a. variable length…

Time homogeneous polynomial processes are Markov processes whose moments can be calculated easily through matrix exponentials. In this work, we develop a notion of time inhomogeneous polynomial processes where the coeffiecients of the…

Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…

Multi-type Markov point processes offer a flexible framework for modelling complex multi-type point patterns where it is pertinent to capture both interactions between points as well as large scale trends depending on observed covariates.…

We treat the class of universal Markov processes on the d-dimensional Euklidean space which do not depend on random. For these, as well as for several subclasses, we prove criteria whether a function f, defined on the positive half-line,…

The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…

Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…

We consider the task of generating draws from a Markov jump process (MJP) between two time-points at which the process is known. Resulting draws are typically termed bridges and the generation of such bridges plays a key role in…

We present a new algorithm for the statistical model checking of Markov chains with respect to unbounded temporal properties, such as reachability and full linear temporal logic. The main idea is that we monitor each simulation run on the…

In this paper we introduce a new approach to discrete-time semi-Markov decision processes based on the sojourn time process. Different characterizations of discrete-time semi-Markov processes are exploited and decision processes are…

Semi-Markov processes play an important role in the effective description of partially accessible systems in stochastic thermodynamics. They occur, for instance, in coarse-graining procedures such as state lumping and when analyzing waiting…

A semi-Markov process method for obtaining general counting statistics for open quantum systems is extended to the scenario of resetting. The simultaneous presence of random resets and wave function collapses means that the quantum jump…

There is a well established theory that links semi-Markov chains having Mittag-Leffler waiting times to time-fractional equations. We here go beyond the semi-Markov setting, by defining some non-Markovian chains whose waiting times,…

In this paper, we consider a type of time-changed Markov process, where the time-change is an inverse killed subordinator. This can be seen as an extension of Chen (Chen, Z., Time fractional equations and probabilistic representation, Chaos…

We consider Markov decision processes (MDP) as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in a single state, or in a given set of…

We present a numerical method to compute expectations of functionals of a piecewise-deterministic Markov process. We discuss time dependent functionals as well as deterministic time horizon problems. Our approach is based on the…

We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these…

In this paper, we study quasi-stationary distributions of nonlinearly perturbed semi-Markov processes in discrete time. This type of distributions is of interest for the analysis of stochastic systems which have finite lifetimes, but are…

This paper studies parametric Markov decision processes (pMDPs), an extension to Markov decision processes (MDPs) where transitions probabilities are described by polynomials over a finite set of parameters. Fixing values for all parameters…