Related papers: Solving Poisson's equation for birth-death chains:…

Given Markov chains and Markov decision processes (MDPs) whose transitions are labelled with non-negative integer costs, we study the computational complexity of deciding whether the probability of paths whose accumulated cost satisfies a…

We study the solution $V$ of the Poisson equation $LV + f=0$ where $L$ is the backward generator of an irreducible (finite) Markov jump process and $f$ is a given centered state function. Bounds on $V$ are obtained using a graphical…

A classical problem for Markov chains is determining their stationary (or steady-state) distribution. This problem has an equally classical solution based on eigenvectors and linear equation systems. However, this approach does not scale to…

The missing data problem pervasively exists in statistical applications. Even as simple as the count data in mortality projections, it may not be available for certain age-and-year groups due to the budget limitations or difficulties in…

The Poisson process is the most elementary continuous-time stochastic process that models a stream of repeating events. It is uniquely characterised by a single parameter called the rate. Instead of a single value for this rate, we here…

For integer valued random variables, the translated Poisson distributions form a flexible family for approximation in total variation, in much the same way that the normal family is used for approximation in Kolmogorov distance. Using the…

We present a perfect simulation algorithm for measures that are absolutely continuous with respect to some Poisson process and can be obtained as invariant measures of birth-and-death processes. Examples include area- and…

A general method to construct recombinant tree approximations for stochastic volatility models is developed and applied to the Heston model for stock price dynamics. In this application, the resulting approximation is a four tuple Markov…

Reversible Markov chains play a central role in stochastic modelling and in algorithms such as Markov chain Monte Carlo (MCMC). Motivated by the fundamental importance of reversibility in classical settings, this paper develops a…

We provide a numerically robust and fast method capable of exploiting the local geometry when solving large-scale stochastic optimisation problems. Our key innovation is an auxiliary variable construction coupled with an inverse Hessian…

The chemical birth-death process, whose chemical master equation (CME) is exactly solvable, is a paradigmatic toy problem often used to get intuition for how stochasticity affects chemical kinetics. In a certain limit, it can be…

We study continuous-time Markov chains on the non-negative integers under mild regularity conditions (in particular, the set of jump vectors is finite and both forward and backward jumps are possible). Based on the so-called flux balance…

A new approach to building explicit time-marching stencil computation schemes for the transient 2D acoustic wave equation is implemented. It is based on using Poisson's formula and its three time level modification combined with polynomial…

A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various…

The simple (linear) birth-and-death process is a widely used stochastic model for describing the dynamics of a population. When the process is observed discretely over time, despite the large amount of literature on the subject, little is…

We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requires $O(1)$…

The paper is concerned with the equilibrium distributions of continuous-time density dependent Markov processes on the integers. These distributions are known typically to be approximately normal, and the approximation error, as measured in…

Policy iteration techniques for multiple-server dispatching rely on the computation of value functions. In this context, we consider the continuous-space M/G/1-FCFS queue endowed with an arbitrarily-designed cost function for the waiting…

Spatial birth-and-death processes with a finite number of particles are obtained as unique solutions to certain stochastic equations. Conditions are given for existence and uniqueness of such solutions, as well as for continuous dependence…

Many important stochastic counting models can be written as general birth-death processes (BDPs). BDPs are continuous-time Markov chains on the non-negative integers and can be used to easily parameterize a rich variety of probability…