Related papers: Thiele's Differential Equation Based on Markov Jum…

Continuous time random walks combining diffusive and ballistic regimes are introduced to describe a class of L\'evy walks on lattices. By including exponentially-distributed waiting times separating the successive jump events of a walker,…

We consider non-standard Markov Decision Processes (MDPs) where the target function is not only a simple expectation of the accumulated reward. Instead, we consider rather general functionals of the joint distribution of terminal state and…

We develop a regression based primal-dual martingale approach for solving finite time horizon MDPs with general state and action space. As a result, our method allows for the construction of tight upper and lower biased approximations of…

We present a new method for simulating Markovian jump processes with time-dependent transitions rates, which avoids the transformation of random numbers by inverting time integrals over the rates. It relies on constructing a sequence of…

Markov chain Monte Carlo methods are central in computational statistics, and typically rely on detailed balance to ensure invariance with respect to a target distribution. Although straightforward to construct by Metropolization, this can…

We investigate the parameter recovery of Markov-switching ordinary differential processes from discrete observations, where the differential equations are nonlinear additive models. This framework has been widely applied in biological…

Markov jump processes are continuous-time stochastic processes which describe dynamical systems evolving in discrete state spaces. These processes find wide application in the natural sciences and machine learning, but their inference is…

We consider the task of generating draws from a Markov jump process (MJP) between two time-points at which the process is known. Resulting draws are typically termed bridges and the generation of such bridges plays a key role in…

Motivated by a risk process with positive and negative premium rates, we consider a real-valued Markov additive process with finitely many background states. This additive process linearly increases or decreases while the background state…

This paper is about the state estimation of timed probabilistic discrete event systems. The main contribution is to propose general procedures for developing state estimation approaches based on artificial neural networks. It is assumed…

Jump Markov linear models consists of a finite number of linear state space models and a discrete variable encoding the jumps (or switches) between the different linear models. Identifying jump Markov linear models makes for a challenging…

Multistate Markov models are a canonical parametric approach for data modeling of observed or latent stochastic processes supported on a finite state space. Continuous-time Markov processes describe data that are observed irregularly over…

We propose a solution to a time-varying variant of Markov Decision Processes which can be used to address decision-theoretic planning problems for autonomous systems operating in unstructured outdoor environments. We explore the time…

This paper introduces time-continuous numerical schemes to simulate stochastic differential equations (SDEs) arising in mathematical finance, population dynamics, chemical kinetics, epidemiology, biophysics, and polymeric fluids. These…

This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying…

In this paper we study the state-feedback stabilization of a discrete-time Markov jump linear system when the observation of the Markov chain of the system, called the Markov state, is time-randomized by another Markov chain. Embedding the…

Studying the behaviour of Markov processes at boundary points of the state space has a long history, dating back all the way to William Feller. With different motivations in mind entrance and exit questions have been explored for different…

We propose a method for approximating solutions to optimization problems involving the global stability properties of parameter-dependent continuous-time autonomous dynamical systems. The method relies on an approximation of the…

The main purpose of the paper is to derive Thiele's differential equation for unit-linked policies in the Heston-Hawkes stochastic volatility model introduced in arXiv:2210.15343. This model is an extension of the well-known Heston model…

We present two main theorems for stochastic processes with a Markov regime-switching model. First, we work on an existence-uniqueness theorem for a Stochastic Differential Delay Equation with Jumps and Regimes (SDDEJRs). Then, we provide…