Related papers: Analysis of Markov Jump Processes under Terminal C…
In this work, we present a general method to establish properties of multi-dimensional continuous-time Markov chains representing stochastic reaction networks. This method consists of grouping states together (via a partition of the state…
For rare events described in terms of Markov processes, truly unbiased estimation of the rare event probability generally requires the avoidance of numerical approximations of the Markov process. Recent work in the exact and…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
The use of stochastic models, in effect piecewise deterministic Markov processes (PDMP), has become increasingly popular especially for the modeling of chemical reactions and cell biophysics. Yet, exact simulation methods, for the…
Optimal queueing control of multi-hop networks remains a challenging problem even in the simplest scenarios. In this paper, we consider a two-hop half-duplex relaying system with random channel connectivity. The relay is equipped with a…
In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior…
We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the…
We present a simulation methodology for Bayesian estimation of rate parameters in Markov jump processes arising for example in stochastic kinetic models. To handle the problem of missing components and measurement errors in observed data,…
Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…
Markov jump process models have many applications across science. Often, these models are defined on a state-space of product form and only one of the components of the process is of direct interest. In this paper, we extend the marginal…
It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…
This paper considers importance sampling for estimation of rare-event probabilities in a specific collection of Markovian jump processes used for e.g. modelling of credit risk. Previous attempts at designing importance sampling algorithms…
In this paper we consider the problem of parameter inference for Markov jump process (MJP) representations of stochastic kinetic models. Since transition probabilities are intractable for most processes of interest yet forward simulation is…
Markov jump processes are continuous-time stochastic processes with a wide range of applications in both natural and social sciences. Despite their widespread use, inference in these models is highly non-trivial and typically proceeds via…
Many real-world phenomena can be modelled as dynamical processes on networks, a prominent example being the spread of infectious diseases such as COVID-19. Mean-field approximations are a widely used tool to analyse such dynamical processes…
This paper motivates the use of random-bridges -- stochastic processes conditioned to take target distributions at fixed timepoints -- in the realm of generative modelling. Herein, random-bridges can act as stochastic transports between two…
This paper considers the problem of remote state estimation for Markov jump linear systems in the presence of uncertainty in the posterior mode probabilities. Such uncertainty may arise when the estimator receives noisy or incomplete…
A $p$-jump process is a piecewise deterministic Markov process with jumps by a factor of $p$. We prove a limit theorem for such processes on the unit interval. Via duality with respect to probability generating functions, we deduce limiting…
Particle splitting methods are considered for the estimation of rare events. The probability of interest is that a Markov process first enters a set $B$ before another set $A$, and it is assumed that this probability satisfies a large…
In this work, we consider the problem of mode clustering in Markov jump models. This model class consists of multiple dynamical modes with a switching sequence that determines how the system switches between them over time. Under different…