Related papers: Efficient sampling of conditioned Markov jump proc…

Markov decision process (MDP) is a decision making framework where a decision maker is interested in maximizing the expected discounted value of a stream of rewards received at future stages at various states which are visited according to…

In this paper, we are interested in the exact simulation of a class of Piecewise Deterministic Markov Processes (PDMP). We show how to perform efficient thinning algorithms depending on the jump rate bound. For different types of jump rate…

We propose a methodology to sample from time-integrated stochastic bridges, namely random variables defined as $\int_{t_1}^{t_2} f(Y(t))dt$ conditioned on $Y(t_1)\!=\!a$ and $Y(t_2)\!=\!b$, with $a,b\in R$. The Stochastic Collocation Monte…

We present a general framework for applying learning algorithms and heuristical guidance to the verification of Markov decision processes (MDPs). The primary goal of our techniques is to improve performance by avoiding an exhaustive…

In this paper, we consider a class of stochastic optimal control problems with risk constraints that are expressed as bounded probabilities of failure for particular initial states. We present here a martingale approach that diffuses a risk…

We propose MDP-GapE, a new trajectory-based Monte-Carlo Tree Search algorithm for planning in a Markov Decision Process in which transitions have a finite support. We prove an upper bound on the number of calls to the generative models…

In simulation-based inferences for partially observed Markov process models (POMP), the by-product of the Monte Carlo filtering is an approximation of the log likelihood function. Recently, iterated filtering [14, 13] has originally been…

This paper concerns the Vertex Reinforced Jump Process (VRJP) and its representations as a Markov process in random environment. We show that all possible representations of the VRJP as a mixture of Markov processes can be expressed in a…

In this paper, we consider risk-sensitive discounted control problem for continuous-time jump Markov processes taking values in general state space. The transition rates of underlying continuous-time jump Markov processes and the cost rates…

Computation of the probability that a random graph is connected is a challenging problem, so it is natural to turn to approximations such as Monte Carlo methods. We describe sequential importance resampling and splitting algorithms for the…

We propose a method to exactly generate bridge run-and-tumble trajectories that are constrained to start at the origin with a given velocity and to return to the origin after a fixed time with another given velocity. The method extends the…

Starting from an iterative and hence numerically easily implementable representation of the thin set of jumps of a c\`{a}dl\`{a}g adapted stochastic process $X$ (including a few applications to the integration with respect to the jump…

Bayesian inference for Markov processes has become increasingly relevant in recent years. Problems of this type often have intractable likelihoods and prior knowledge about model rate parameters is often poor. Markov Chain Monte Carlo…

Flowgraph models provide an alternative approach in modeling a multi-state stochastic process. One of the most widely used stochastic processes that have many real-world applications especially in actuarial models is the Markov jump process…

The problem of optimising functions with intractable gradients frequently arise in machine learning and statistics, ranging from maximum marginal likelihood estimation procedures to fine-tuning of generative models. Stochastic approximation…

In this paper we discuss weak convergence of continuous-time Markov chains to a non-symmetric pure jump process. We approach this problem using Dirichlet forms as well as semimartingales. As an application, we discuss how to approximate a…

Many problems in the physical sciences, machine learning, and statistical inference necessitate sampling from a high-dimensional, multi-modal probability distribution. Markov Chain Monte Carlo (MCMC) algorithms, the ubiquitous tool for this…

The Reversible Jump algorithm is one of the most widely used Markov chain Monte Carlo algorithms for Bayesian estimation and model selection. A generalized multiple-try version of this algorithm is proposed. The algorithm is based on…

Markov chain Monte Carlo is a class of algorithms for drawing Markovian samples from high-dimensional target densities to approximate the numerical integration associated with computing statistical expectation, especially in Bayesian…

We present a simulation methodology for Bayesian estimation of rate parameters in Markov jump processes arising for example in stochastic kinetic models. To handle the problem of missing components and measurement errors in observed data,…