Related papers: Piecewise deterministic Markov processes in biolog…
The work treats systems combining slow and fast motions depending on each other where fast motions are perturbations of families of either dynamical systems or Markov processes with freezed slow variable. In the first case we consider…
New algorithms for computing of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes are presented. The algorithms are based on special techniques of sequential phase space reduction, which can…
Consider a branching Markov process, $X = (X(t), t \ge 0)$, with non-local branching mechanism. Studying the asymptotic behaviour of the moments of X has recently received attention in the literature [6, 7] due to the importance of these…
We study the large-time behavior of a class of periodically driven macroscopic systems. We find, for a certain range of the parameters of either the system or the driving fields, the time-averaged asymptotic behavior effectively is that of…
This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs). Typically, computing the optimal policy of a POMDP is computationally intractable. We use lattice program- ming…
It has been established under very general conditions that the ergodic properties of Markov processes are inherited by their conditional distributions given partial information. While the existing theory provides a rather complete picture…
The purpose of time series analysis via mechanistic models is to reconcile the known or hypothesized structure of a dynamical system with observations collected over time. We develop a framework for constructing nonlinear mechanistic models…
Using the age-structure formalism, we definitely establish connections between semi-Markov processes and the dynamics of open quantum systems that satisfy the Markov quantum master equations. A generalized Feynman-Kac formula of the…
Scientific inference involves obtaining the unknown properties or behavior of a system in the light of what is known, typically, without changing the system. Here we propose an alternative to this approach: a system can be modified in a…
Applications of first passage times in stochastic processes arise across a wide range of length and time scales in biological settings. After an initial technical overview, we survey representative applications and their corresponding…
In recent work, Baez, Fong and the author introduced a framework for describing Markov processes equipped with a detailed balanced equilibrium as open systems of a certain type. These `open Markov processes' serve as the building blocks for…
There is much interest in using partially observable Markov decision processes (POMDPs) as a formal model for planning in stochastic domains. This paper is concerned with finding optimal policies for POMDPs. We propose several improvements…
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision…
Stochastic processes have found numerous applications in science, as they are broadly used to model a variety of natural phenomena. Due to their intrinsic randomness and uncertainty, they are, however, difficult to characterize. Here, we…
Hidden tree Markov models allow learning distributions for tree structured data while being interpretable as nondeterministic automata. We provide a concise summary of the main approaches in literature, focusing in particular on the…
This paper introduces Switching Processes, called SP. Their constructions are inspired by the PDMP's ones (PDMP stands for Piecewise Deterministic Markov Process). A Markov process, called the intrinsic process, replaces the PDMP's flow.…
Regular and singular parts of asymptotic expansions of semi-Markov random evolutions are given. Regularity of boundary conditions is shown. An algorithm for calculation of initial conditions is proposed.
Piecewise diffusion Markov processes (PDifMPs) form a versatile class of stochastic hybrid systems that combine continuous diffusion processes with discrete event-driven dynamics, enabling flexible modelling of complex real-world hybrid…
We construct a family of genealogy-valued Markov processes that are induced by a continuous-time Markov population process. We derive exact expressions for the likelihood of a given genealogy conditional on the history of the underlying…
Monte Carlo methods -- such as Markov chain Monte Carlo (MCMC) and piecewise deterministic Markov process (PDMP) samplers -- provide asymptotically exact estimators of expectations under a target distribution. There is growing interest in…