Related papers: Thiele's Differential Equation Based on Markov Jum…
Modern methods of simulating molecular systems are based on the mathematical theory of Markov operators with a focus on autonomous equilibrated systems. However, non-autonomous physical systems or non-autonomous simulation processes are…
We study a finite-horizon covariance steering problem for discrete-time Markov jump linear systems (MJLS) with both state- and control-dependent multiplicative noise. The objective is to minimize a quadratic running cost while steering the…
Rate-independent systems allow for solutions with jumps that need additional modeling. Here we suggest a formulation that arises as limit of viscous regularization of the solutions in the extended state space. Hence, our parametrized metric…
Multi-state survival analysis considers several potential events of interest along a disease pathway. Such analyses are crucial to model complex patient trajectories and are increasingly being used in epidemiological and health economic…
A $p$-jump process is a piecewise deterministic Markov process with jumps by a factor of $p$. We prove a limit theorem for such processes on the unit interval. Via duality with respect to probability generating functions, we deduce limiting…
In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent…
We consider a stochastic functional delay differential equation, namely an equation whose evolution depends on its past history as well as on its present state, driven by a pure diffusive component plus a pure jump Poisson compensated…
For a continuous-time Markov process, we characterize the law of the first jump location when started from an arbitrary initial distribution, in terms of the invariant distribution of an auxiliary Markov process. This could be of interest…
The solution convergence of Markov Decision Processes (MDPs) can be accelerated by prioritized sweeping of states ranked by their potential impacts to other states. In this paper, we present new heuristics to speed up the solution…
This paper studies continuous-time Markov decision processes under the risk-sensitive average cost criterion. The state space is a finite set, the action space is a Borel space, the cost and transition rates are bounded, and the…
We address a class of Markov jump linear systems that are characterized by the underlying Markov process being time-inhomogeneous with a priori unknown transition probabilities. Necessary and sufficient conditions for uniform stochastic…
The Lindblad equation describes the time evolution of a density matrix of a quantum mechanical system. Stationary solutions are obtained by time-averaging the solution, which will in general depend on the initial state. We provide an…
We compute the stationary distribution of a continuous-time Markov chain which is constructed by gluing together two finite, irreducible Markov chains by identifying a pair of states of one chain with a pair of states of the other and…
Markov cohort state-transition models have been the standard approach for simulating the prognosis of patients or, more generally, the life trajectories of individuals over a time period. Current approaches for estimating the variance of a…
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
Survival models are a popular tool for the analysis of time to event data with applications in medicine, engineering, economics, and many more. Advances like the Cox proportional hazard model have enabled researchers to better describe…
In this paper we introduce a bivariate distribution on $\mathbb{R}_{+} \times \mathbb{N}$ arising from a single underlying Markov jump process. The marginal distributions are phase-type and discrete phase-type distributed, respectively,…
In multi-state life insurance, an adequate balance between analytic tractability, computational efficiency, and statistical flexibility is of great importance. This might explain the popularity of Markov chain modelling, where matrix…
A large class of linear memory differential equations in one dimension, where the evolution depends on the whole history, can be equivalently described as a projection of a Markov process living in a higher dimensional space. Starting with…