Related papers: Bounds for the expected value of one-step processe…
In this note we consider a Markov chain formed by a finite system of interacting birth-and-death processes on a finite state space. We study an asymptotic behaviour of the Markov chain as its state space becomes large. In particular, we…
Studies of fixation dynamics in Markov processes predominantly focus on the mean time to absorption. This may be inadequate if the distribution is broad and skewed. We compute the distribution of fixation times in one-step birth-death…
This paper investigates the position (state) distribution of the single step binomial (multi-nomial) process on a discrete state / time grid under the assumption that the velocity process rather than the state process is Markovian. In this…
We present two data-driven procedures to estimate the transition density of an homogeneous Markov chain. The first yields to a piecewise constant estimator on a suitable random partition. By using an Hellinger-type loss, we establish…
We are studying stationary random processes with conditional polynomial moments that allow a continuous path modification. Processes with continuous path modification, are important because they are relatively easy to simulate. One does not…
Focusing on stochastic systems arising in mean-field models, the systems under consideration belong to the class of switching diffusions, in which continuous dynamics and discrete events coexist and interact. The discrete events are modeled…
Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this paper, we propose a novel solution to this decision-theoretic planning problem that directly…
Non-Markovian models have great expressive power, at the cost of complex analysis of the stochastic process. The method of Stochastic State Classes (SSCs) derives closed-form analytical expressions for the joint Probability Density…
This paper studies birth and death processes in interactive random environments where the birth and death rates and the dynamics of the state of the environment are dependent on each other. Two models of a random environment are considered:…
Markov switching models are a popular family of models that introduces time-variation in the parameters in the form of their state- or regime-specific values. Importantly, this time-variation is governed by a discrete-valued latent…
We consider a continuous time Markov process on $\mathbb{N}_0$ which can be interpreted as generalized alternating birth-death process in a non-autonomous random environment. Depending on the status of the environment the process either…
In order to describe the extremal behaviour of some stochastic process $X$, approaches from univariate extreme value theory are typically generalized to the spatial domain. In particular, generalized peaks-over-threshold approaches allow…
This paper presents a new view of household epidemic models, where the interaction between the households is of mean field type. We thus obtain in the limit of infinitely many households a nonlinear Markov process solution of a McKean -…
When modelling metapopulation dynamics, the influence of a single patch on the metapopulation depends on the number of individuals in the patch. Since there is usually no obvious natural upper limit on the number of individuals in a patch,…
We present metrics for measuring the similarity of states in a finite Markov decision process (MDP). The formulation of our metrics is based on the notion of bisimulation for MDPs, with an aim towards solving discounted infinite horizon…
We study continuous-time birth-death type processes, where individuals have independent and identically distributed lifetimes, according to a random variable Q, with E[Q]=1, and where the birth rate if the population is currently in state…
The main substance of the paper concerns the growth rate and the classification (ergodicity, transience) of a family of random trees. In the basic model, new edges appear according to a Poisson process of parameter $\lambda$ and leaves can…
This paper studies the spread dynamics of a stochastic SIRS epidemic model with nonlinear incidence and varying population size, which is formulated as a piecewise deterministic Markov process. A threshold dynamic determined by the basic…
We consider an asexually reproducing population on a finite type space whose evolution is driven by exponential birth, death and competition rates, as well as the possibility of mutation at a birth event. On the individual-based level this…
This paper develops a new class of conditional Markov jump processes with regime switching and paths dependence. The key novel feature of the developed process lies on its ability to switch the transition rate as it moves from one state to…