Related papers: Diffusion models and steady-state approximations f…
This paper considers continuous-time block-monotone Markov chains (BMMCs) and their block-augmented truncations. We first introduce the block monotonicity and block-wise dominance relation for continuous-time Markov chains, and then provide…
We use an Ornstein--Uhlenbeck (OU) process to approximate the queue length process in a $GI/GI/n+M$ queue. This one-dimensional diffusion model is able to produce accurate performance estimates in two overloaded regimes: In the first…
We study fast / slow systems driven by a fractional Brownian motion $B$ with Hurst parameter $H\in (\frac 13, 1]$. Surprisingly, the slow dynamic converges on suitable timescales to a limiting Markov process and we describe its generator.…
Continuous-time Bayesian networks (CTBNs) are graphical representations of multi-component continuous-time Markov processes as directed graphs. The edges in the network represent direct influences among components. The joint rate matrix of…
We consider a collection of Markov chains that model the evolution of multitype biological populations. The state space of the chains is the positive orthant, and the boundary of the orthant is absorbing representing the extinction states…
We study the optimal scheduling problem for a Markovian multiclass queueing network with abandonment in the Halfin--Whitt regime, under the long run average (ergodic) risk sensitive cost criterion. The objective is to prove asymptotic…
A Markov state model is a powerful tool that can be used to track the evolution of populations of configurations in an atomistic representation of a protein. For a coarse-grained linear chain model with discontinuous interactions, the…
A variety of enhanced statistical and numerical methods are now routinely used to extract comprehensible and relevant thermodynamic information from the vast amount of complex, high-dimensional data obtained from intensive molecular…
Consideration is given to the three different analytical methods for the computation of upper bounds for the rate of convergence to the limiting regime of one specific class of (in)homogeneous continuous-time Markov chains. This class is…
We prove the existence of limiting distributions for a large class of Markov chains on a general state space in a random environment. We assume suitable versions of the standard drift and minorization conditions. In particular, the system…
We investigate the convergence to (quasi--)equilibrium of a density dependent Markov chain in~${\mathbb Z}^d$, whose drift satisfies a system of ordinary differential equations having an attractive fixed point. For a sequence of such…
Continuous Time Markov Chain (CMTC) is widely used to describe and analyze systems in several knowledge areas. Steady state availability is one important analysis that can be made through Markov chain formalism that allows researchers…
We investigate the Brownian diffusion of particles in one spatial dimension and in the presence of finite regions within which particles can either evaporate or be reset to a given location. For open boundary conditions, we highlight the…
We provide a general steady-state diffusion approximation result which bounds the Wasserstein distance between the reversible measure $\mu$ of a diffusion process and the measure $\nu$ of an approximating Markov chain. Our result is…
We introduce the {\Delta}(i)/GI/1 queue, a new queueing model. In this model, customers from a given population independently sample a time to arrive from some given distribution F. Thus, the arrival times are an ordered statistics, and the…
In this paper we study the uniform stability properties of two classes of parallel server networks with multiple classes of jobs and multiple server pools of a tree topology. These include a class of networks with a single non-leaf server…
We consider a class of discrete time Markov chains with state space [0,1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then…
A central task in many applications is reasoning about processes that change in a continuous time. The mathematical framework of Continuous Time Markov Processes provides the basic foundations for modeling such systems. Recently, Nodelman…
We present some new results on sample path optimality for the ergodic control problem of a class of non-degenerate diffusions controlled through the drift. The hypothesis most often used in the literature to ensure the existence of an a.s.…
In this paper, we quantify some known approximation to the Curie-Weiss model via applying the Stein method to the Markov chain whose stationary distribution coincides with Curie-Weiss model.