Related papers: Marginal process framework: A model reduction tool…
Markov decision processes are a ubiquitous formalism for modelling systems with non-deterministic and probabilistic behavior. Verification of these models is subject to the famous state space explosion problem. We alleviate this problem by…
A hidden Markov process is a well known concept in information theory and is used for a vast range of applications such as speech recognition and error correction. We bridge between two disciplines, experimental physics and advanced…
This paper is dedicated to the numerical study of the optimization of an industrial launcher integration process. It is an original case of inventory-production system where a calendar plays a crucial role. The process is modeled using the…
Adaptive and interacting Markov chain Monte Carlo algorithms (MCMC) have been recently introduced in the literature. These novel simulation algorithms are designed to increase the simulation efficiency to sample complex distributions.…
Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artificial intelligence (AI). MDPs do, however, come with the…
Mean-field integro-differential equations are studied in an abstract framework, through couplings of the corresponding stochastic processes. In the perturbative regime, the equation is proven to admit a unique equilibrium, toward which the…
A constrained Markov decision process (CMDP) approach is developed for response-adaptive procedures in clinical trials with binary outcomes. The resulting CMDP class of Bayesian response -- adaptive procedures can be used to target a…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
In computational system biology, the mesoscopic model of reaction-diffusion kinetics is described by a continuous time, discrete space Markov process. To simulate diffusion stochastically, the jump coefficients are obtained by a…
We consider continuous-time, finite-horizon, optimal quadratic control of semi-Markov jump linear systems (S-MJLS), and develop principled approximations through Markov-like representations for the holding-time distributions. We adopt a…
We show the variational convergence of an irreversible Markov jump process describing a finite stochastic particle system to the solution of a countable infinite system of deterministic time-inhomogeneous quadratic differential equations…
In this paper we discuss weak convergence of continuous-time Markov chains to a non-symmetric pure jump process. We approach this problem using Dirichlet forms as well as semimartingales. As an application, we discuss how to approximate a…
Conventional joint modeling approaches generally characterize the relationship between longitudinal biomarkers and discrete event occurrences within terminal, recurring or competing risk settings, thereby offering a limited representation…
We present a general framework for applying learning algorithms and heuristical guidance to the verification of Markov decision processes (MDPs). The primary goal of our techniques is to improve performance by avoiding an exhaustive…
This paper considers importance sampling for estimation of rare-event probabilities in a specific collection of Markovian jump processes used for e.g. modelling of credit risk. Previous attempts at designing importance sampling algorithms…
We study Markov-modulated affine processes (abbreviated MMAPs), a class of Markov processes that are created from affine processes by allowing some of their coefficients to be a function of an exogenous Markov process. MMAPs allow for…
Diffusive approximations of Markov jump processes often fail to accurately capture large fluctuations. This is confounding, as the rare events triggered by these large fluctuations, such as the failure of electronic memories, are often the…
Stochastic kinetic models (SKMs) are increasingly used to account for the inherent stochasticity exhibited by interacting populations of species in areas such as epidemiology, population ecology and systems biology. Species numbers are…
We establish an abstract, effective, exponential large deviations type estimate for Markov systems satisfying a weaker form of mixing. We employ this result to derive such estimates, as well as a central limit theorem, for the skew product…
We present a Markov approximation for jump-diffusions whose jump part consists in a Hawkes process with intensity driven by a general (possibly non-monotone) kernel. Under minimal integrability conditions, the kernel can be approximated by…