Related papers: Bounds for the expected value of one-step processe…

We study the asymptotic behaviour of Markov processes on large weighted Erdos-Renyi graphs where the transition rates of the vertices are only influenced by the state of their neighbours and the corresponding weight on the edges. We find…

We study Markov population processes on large graphs, with the local state transition rates of a single vertex being linear function of its neighborhood. A simple way to approximate such processes is by a system of ODEs called the…

Mean-field models approximate large stochastic systems by simpler differential equations that are supposed to approximate the mean of the larger system. It is generally assumed that as the stochastic systems get larger (i.e., more people or…

We study a Markov decision problem in which the state space is the set of finite marked point configurations in the plane, the actions represent thinnings, the reward is proportional to the mark sum which is discounted over time, and the…

In this paper we propose a new method for approximating the nonstationary moment dynamics of one dimensional Markovian birth-death processes. By expanding the transition probabilities of the Markov process in terms of Poisson-Charlier…

This paper is a continuation of the study on the stability speed for Markov processes. It extends the previous study of the ergodic convergence speed to the non-ergodic one, in which the processes are even allowed to be explosive or having…

The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…

We study ergodic properties of a class of Markov-modulated general birth-death processes under fast regime switching. The first set of results concerns the ergodic properties of the properly scaled joint Markov process with a parameter that…

We study the computational complexity of central analysis problems for One-Counter Markov Decision Processes (OC-MDPs), a class of finitely-presented, countable-state MDPs. OC-MDPs are equivalent to a controlled extension of (discrete-time)…

Deterministic models of vegetation often summarize, at a macroscopic scale, a multitude of intrinsically random events occurring at a microscopic scale. We bridge the gap between these scales by demonstrating convergence to a mean-field…

This paper studies a large number of homogeneous Markov decision processes where the transition probabilities and costs are coupled in the empirical distribution of states (also called mean-field). The state of each process is not known to…

We describe an exact approach for calculating transition probabilities and waiting times in finite-state discrete-time Markov processes. All the states and the rules for transitions between them must be known in advance. We can then…

Epidemic dynamics in a stochastic network of interacting epidemic centers is considered. The epidemic and migration processes are modelled by Markov's chains. Explicit formulas for probability distribution of the migration process are…

In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent…

We consider a finite number of $N$ statistically equal agents, each moving on a finite set of states according to a continuous-time Markov Decision Process (MDP). Transition intensities of the agents and generated rewards depend not only on…

We provide a mean-field description for a leader-follower dynamics with mass transfer among the two populations. This model allows the transition from followers to leaders and vice versa, with scalar-valued transition rates depending…

We study normal approximations for a class of discrete-time occupancy processes, namely, Markov chains with transition kernels of product Bernoulli form. This class encompasses numerous models which appear in the complex networks…

In this paper we present elementary computations for some Markov modulated counting processes, also called counting processes with regime switching. Regime switching has become an increasingly popular concept in many branches of science. In…

The bike sharing systems are fast increasing as a public transport mode in urban short trips, and have been developed in many major cities around the world. A major challenge in the study of bike sharing systems is that large-scale and…

In this paper, we consider a class of inhomogeneous semi-Markov processes directly based on intensity processes for marked point processes. We show that this class satisfies the semi-Markov properties defined elsewhere in the literature. We…