Related papers: Birth-death-suppression Markov process and wildfir…
Boundary-catalytic branching processes describe a broad class of natural phenomena where the population of diffusing particles grows due to their spontaneous binary branching (e.g., division, fission or splitting) on a catalytic boundary…
Experiments, in particular on biological systems, typically probe lower-dimensional observables which are projections of high-dimensional dynamics. In order to infer consistent models capturing the relevant dynamics of the system, it is…
We investigate how a catastrophic event (modeled as a temporary fall of the reproduction rate) increases the extinction probability of an isolated self-regulated stochastic population. Using a variant of the Verhulst logistic model as an…
We propose a Markovian stochastic approach to model the spread of a SARS-CoV-2-like infection within a closed group of humans. The model takes the form of a Partially Observable Markov Decision Process (POMDP), whose states are given by the…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…
Asymptotic properties of Markov Processes, such as steady state probabilities or hazard rate for absorbing states can be efficiently calculated by means of linear algebra even for large-scale problems. This paper discusses the methods for…
We present a stochastic model of population dynamics exploiting cross-sectional data in trend analysis and forecasts for groups and cohorts of a population. While sharing the convenient features of classic Markov models, it alleviates the…
We describe a general approach to the construction of a state evolution corresponding to the Markov generator of a spatial birth-and-death dynamics in $\mathbb{R}^d$. We present conditions on the birth-and-death intensities which are…
Lattice birth-and-death Markov dynamics of particle systems with spins from the set of non-negative integers are constructed as unique solutions to certain stochastic equations. Pathwise uniqueness, strong existence, Markov property and…
An infinite system of point particles placed in $\mathds{R}^d$ is studied. The particles are of two types; they perform random walks in the course of which those of distinct types repel each other. The interaction of this kind induces an…
Critically ill patients in regular wards are vulnerable to unanticipated adverse events which require prompt transfer to the intensive care unit (ICU). To allow for accurate prognosis of deteriorating patients, we develop a novel…
We consider finite model approximations of discrete-time partially observed Markov decision processes (POMDPs) under the discounted cost criterion. After converting the original partially observed stochastic control problem to a fully…
In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
We consider a class of birth-and-death processes describing a population made of $d$ sub-populations of different types which interact with one another. The state space is $\mathbb{Z}_+^d$ (unbounded). We assume that the population goes…
Motivated by the study of a parasite infection in a cell line, we introduce a general class of Markov processes for the modelling of population dynamics. The population process evolves as a diffusion with positive jumps whose rate is a…
The reduced Markov branching process is a stochastic model for the genealogy of an unstructured biological population. Its limit behavior in the critical case is well studied for the Zolotarev-Slack regularity parameter $\alpha\in(0,1]$. We…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
Studies of fixation dynamics in Markov processes predominantly focus on the mean time to absorption. This may be inadequate if the distribution is broad and skewed. We compute the distribution of fixation times in one-step birth-death…