Related papers: Full classification of dynamics for one-dimensiona…
Let $(X_t)_{t \geq 0}$ be a continuous time Markov process on some metric space $M,$ leaving invariant a closed subset $M_0 \subset M,$ called the {\em extinction set}. We give general conditions ensuring either "Stochastic persistence"…
For downward skip-free continuous-time Markov chains on non-negative integers stopped at zero, existence of a quasi-stationary distribution is studied. The scale function for these processes is introduced and the boundary is classified by a…
Studies of fixation dynamics in Markov processes predominantly focus on the mean time to absorption. This may be inadequate if the distribution is broad and skewed. We compute the distribution of fixation times in one-step birth-death…
We describe an exact approach for calculating transition probabilities and waiting times in finite-state discrete-time Markov processes. All the states and the rules for transitions between them must be known in advance. We can then…
We propose a novel nonparametric approach for linking covariates to Continuous Time Markov Chains (CTMCs) using the mathematical framework of Reproducing Kernel Hilbert Spaces (RKHS). CTMCs provide a robust framework for modeling…
This article studies the quasi-stationary behaviour of population processes with unbounded absorption rate, including one-dimensional birth and death processes with catastrophes and multi-dimensional birth and death processes, modeling…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
We consider consistent dynamics for non-intersecting birth and death chains, originating from dualities of stochastic coalescing flows and one dimensional orthogonal polynomials. As corollaries, we obtain unified and simple probabilistic…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic…
A divide-and-conquer approach to analyzing Markov chains (MCs) is not utilized as widely as it could be, despite its potential benefits. One primary reason for this is the fact that most MC decomposition approaches involve a complex and…
Spatial multiscale methods have established themselves as useful tools for extending the length scales accessible by conventional statics (i.e., zero temperature molecular dynamics). Recently, extensions of these methods, such as the…
We consider the task of learning a parametric Continuous Time Markov Chain (CTMC) sequence model without examples of sequences, where the training data consists entirely of aggregate steady-state statistics. Making the problem harder, we…
In this article, we propose a novel and general dimension-hopping MCMC methodology that can update all the parameters as well as the number of parameters simultaneously using simple deterministic transformations of some low-dimensional…
In this paper, we propose a novel stochastic process that serves as a natural discrete-time counterpart to the continuous-time model known as the ``Poisson hyperbolic staircase'' proposed by Levikson et al. (1999), and clarify its…
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…
Motivated by queues with many servers, we study Brownian steady-state approximations for continuous time Markov chains (CTMCs). Our approximations are based on diffusion models (rather than a diffusion limit) whose steady-state, we prove,…
Hoeffding's inequality is a fundamental tool widely applied in probability theory, statistics, and machine learning. In this paper, we establish Hoeffding's inequalities specifically tailored for an irreducible and positive recurrent…
We derive sufficient and necessary optimality conditions in terms of a stochastic maximum principle (SMP) for controls associated with cost functionals of mean-field type, under dynamics driven by a class of Markov chains of mean-field type…
We compute the stationary distribution of a continuous-time Markov chain which is constructed by gluing together two finite, irreducible Markov chains by identifying a pair of states of one chain with a pair of states of the other and…