Related papers: Timed Comparisons of Semi-Markov Processes
In this paper, we obtain a Lamperti type representation for real-valued self-similar Markov processes, killed at their hitting time of zero. Namely, we represent real-valued self-similar Markov processes as time changed multiplicative…
We study very simple sorting algorithms based on a probabilistic comparator model. In our model, errors in comparing two elements are due to (1) the energy or effort put in the comparison and (2) the difference between the compared…
In this paper we discuss weak convergence of continuous-time Markov chains to a non-symmetric pure jump process. We approach this problem using Dirichlet forms as well as semimartingales. As an application, we discuss how to approximate a…
Many applications in medical statistics as well as in other fields can be described by transitions between multiple states (e.g. from health to disease) experienced by individuals over time. In this context, multi-state models are a popular…
We present a new method for simulating Markovian jump processes with time-dependent transitions rates, which avoids the transformation of random numbers by inverting time integrals over the rates. It relies on constructing a sequence of…
We obtain weak rates for approximation of an integral functional of a Markov process by integral sums. An assumption on the process is formulated only in terms of its transition probability density, and, therefore, our approach is not…
Partially observable Markov decision processes (POMDPs) provide an elegant mathematical framework for modeling complex decision and planning problems in stochastic domains in which states of the system are observable only indirectly, via a…
We construct a large class of non-Markovian master equations that describe the dynamics of open quantum systems featuring strong memory effects, which relies on a quantum generalization of the concept of classical semi-Markov processes.…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
We introduce the minimal maximally predictive models ({\epsilon}-machines) of processes generated by certain hidden semi-Markov models. Their causal states are either hybrid discrete-continuous or continuous random variables and…
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…
Motivated by wide-ranging applications such as video delivery over networks using Multiple Description Codes, congestion control, and inventory management, we study the state-tracking of a Markovian random process with a known transition…
The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…
We study the asymptotic properties, in the weak sense, of regenerative processes and Markov renewal processes. For the latter, we derive both renewal-type results, also concerning the related counting process, and ergodic-type ones,…
We study continuous time Markov processes on graphs. The notion of frequency is introduced, which serves well as a scaling factor between any Markov time of a continuous time Markov process and that of its jump chain. As an application, we…
We provide a large class of quantum evolution governed by the memory kernel master equation. This class defines quantum analog of so called semi-Markov classical stochastic evolution. In this Letter for the first time we provide a proper…
In this paper we propose two behavioral distances that support approximate reasoning on Stochastic Markov Models (SMMs), that are continuous-time stochastic transition systems where the residence time on each state is described by a generic…
We describe an exact approach for calculating transition probabilities and waiting times in finite-state discrete-time Markov processes. All the states and the rules for transitions between them must be known in advance. We can then…
Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…
Using a newly introduced connection between the local and non-local description of open quantum system dynamics, we investigate the relationship between these two characterisations in the case of quantum semi-Markov processes. This class of…