Related papers: Bounds for the expected value of one-step processe…
We derive the exact solution of a one-dimensional Markov functional model with log-normally distributed interest rates in discrete time. The model is shown to have two distinct limiting states, corresponding to small and asymptotically…
We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…
We show that the joint probability generating function of the stationary measure of a finite state asymmetric exclusion process with open boundaries can be expressed in terms of joint moments of Markov processes called quadratic harnesses.…
The contact process is a particular case of birth-and-death processes on infinite particle configurations. We consider the contact models on locally compact separable metric spaces. We prove the existence of a one-parameter set of invariant…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
We introduce an analytical model for population dynamics with intra-specific competition, mutation and assortative mating as basic ingredients. The set of equations that describes the time evolution of population size in a mean-field…
We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are…
Given a large number of homogeneous players that are distributed across three possible states, we consider the problem in which these players have to control their transition rates, while minimizing a cost. The optimal transition rates are…
Scaling limits for continuous-time branching processes with discrete state space are provided as the initial state tends to infinity. Depending on the finiteness or non-finiteness of the mean and/or the variance of the offspring…
We establish new conditions for obtaining uniform bounds on the moments of discrete-time stochastic processes. Our results require a weak negative drift criterion along with a state-dependent restriction on the sizes of the one-step jumps…
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…
In this paper, we consider semi-Markov processes whose transition times and transition probabilities depend on a small parameter $\varepsilon$. Understanding the asymptotic behavior of such processes is needed in order to study the…
We consider the convergence of a continuous-time Markov chain approximation X^h, h>0, to an R^d-valued Levy process X. The state space of X^h is an equidistant lattice and its Q-matrix is chosen to approximate the generator of X. In…
A new mean field statistical mechanics model of two interacting groups of spins is introduced and the phase transition studied in terms of their relative size. A jump of the average magnetization is found for large values of the mutual…
The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the…
We view the classical Lindeberg principle in a Markov process setting to establish a probability approximation framework by the associated It\^{o}'s formula and Markov operator. As applications, we study the error bounds of the following…
The ordinary contact process is used to model the spread of a disease in a population. In this model, each infected individual waits an exponentially distributed time with parameter 1 before becoming healthy. In this paper, we introduce and…
This paper presents a state representation framework for Markov decision processes (MDPs) that can be learned solely from state trajectories, requiring neither reward signals nor the actions executed by the agent. We propose learning the…
A birth-death process is a continuous-time Markov chain that counts the number of particles in a system over time. In the general process with $n$ current particles, a new particle is born with instantaneous rate $\lambda_n$ and a particle…
Consider the batch-arrival $GI^X/M/c/N$ model with $c$ servers, general inter-arrival batch times, finite buffer, and exponential service times. Inter-arrival batch times, batch sizes, and service times are $i.i.d.$ and independent of each…