Related papers: Diffusion models and steady-state approximations f…
Consider a sequence $P_n$ of positive recurrent transition matrices or kernels that approximate a limiting infinite state matrix or kernel $P_{\infty}$. Such approximations arise naturally when one truncates an infinite state Markov chain…
We study the infinite-horizon average (ergodic) risk sensitive control problem for diffusion processes under a general structural hypothesis: there is a partition of state space into two subsets, where the controlled diffusion process…
We consider triangular arrays of Markov chains that converge weakly to a diffusion process. Edgeworth type expansions of third order for transition densities are proved. This is done for time horizons that converge to 0. For this purpose we…
A new approach to the steady state detection in the uniformization method of solving continuous time Markov chains is introduced. The method is particularly useful in solving inhomogenous CTMC's in multiple steps, where the desired error…
We develop the first exact Bayesian methodology for the problem of inference in discretely observed regime switching diffusions. Switching diffusion models extend ordinary diffusions by allowing for jumps in instantaneous drift and…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
This is an expository review paper illustrating the ``martingale method'' for proving many-server heavy-traffic stochastic-process limits for queueing models, supporting diffusion-process approximations. Careful treatment is given to an…
This study explores a Gaussian quasi-likelihood approach for estimating parameters of diffusion processes with Markovian regime switching. Assuming the ergodicity under high-frequency sampling, we will show the asymptotic normality of the…
Discrete diffusion models based on continuous-time Markov chains (CTMCs) have shown strong performance on language and discrete data generation, yet existing approaches typically parameterize the reverse rate matrix monolithically --…
This paper provides an introduction to the Stein method framework in the context of steady-state diffusion approximations. The framework consists of three components: the Poisson equation and gradient bounds, generator coupling, and moment…
Ramaswami showed recently that standard Brownian motion arises as the limit of a family of Markov-modulated linear fluid processes. We pursue this analysis with a fluid approximation for Markov-modulated Brownian motion. Furthermore, we…
The stochastic thermodynamics of a dilute, well-stirred mixture of chemically-reacting species is built on the stochastic trajectories of reaction events obtained from the Chemical Master Equation. However, when the molecular populations…
In the presence of quantum measurements with direct photon detection the evolution of open quantum systems is usually described by stochastic master equations with jumps. Heuristically, from these equations one can obtain diffusion models…
We study the estimation of the value function for continuous-time Markov diffusion processes using a single, discretely observed ergodic trajectory. Our work provides non-asymptotic statistical guarantees for the least-squares…
A single queueing system with time-dependent exponentially distributed arrival processes and exponential machine processes (Kendall notation $M_t/M_t/1$) is analyzed. Modeling the time evolution for the discrete queue-length distribution by…
Motivated by call center practice, we propose a tractable model for $\mbox{GI}/\mbox{GI}/n+\mbox{GI}$ queues in the efficiency-driven (ED) regime. We use a one-dimensional diffusion process to approximate the virtual waiting time process…
Time bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving…
Mathematically modelling diffusive and advective transport of particles in heterogeneous layered media is important to many applications in computational, biological and medical physics. While deterministic continuum models of such…
We introduce the Space-Time Markov Chain Approximation (STMCA) for a general diffusion process on a finite metric graph $\Gamma$. The STMCA is a doubly asymmetric (in both time and space) random walk defined on a subdivisions of $\Gamma$,…
Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential…