Related papers: Stationary analysis of certain Markov-modulated re…
A possibly time-dependent transition intensity matrix or generator $(Q(t))$ characterizes the law of a Markov jump process (MP). For a time homogeneous MP, the transition probability matrix (TPM) can be expressed as a matrix exponential of…
In this paper, we develop a more general framework of block-structured Markov processes in the queueing study of blockchain systems, which can provide analysis both for the stationary performance measures and for the sojourn times of any…
In stochastic models for queues and their networks, random events evolve in time. A process for their backward evolution is referred to as a time reversed process. It is often greatly helpful to view a stochastic model from two different…
We consider a class of stochastic dynamical systems, called piecewise deterministic Markov processes, with states $(x, \s)\in \O\times \G$, $\O$ being a region in $\bbR^d$ or the $d$--dimensional torus, $\G$ being a finite set. The…
Diffusion processes have been widely used for approximations in the queueing theory. There are different types of diffusion approximations. Among them, we are interested in those obtained through limits of a sequence of models which…
This paper studies a scheduling control problem for a single-server multiclass queueing network in heavy traffic, operating in a changing environment. The changing environment is modeled as a finite state Markov process that modulates the…
Direct simulation of biomolecular dynamics in thermal equilibrium is challenging due to the metastable nature of conformation dynamics and the computational cost of molecular dynamics. Biased or enhanced sampling methods may improve the…
This paper introduces a general model of a single-lane roundabout, represented as a circular lattice that consists of $L$ cells, with Markovian traffic dynamics. Vehicles enter the roundabout via on-ramp queues that have stochastic arrival…
Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the…
In this paper, we are interested in investigating the perturbation bounds for the stationary distributions for discrete-time or continuous-time Markov chains on a countable state space. For discrete-time Markov chains, two new norm-wise…
We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the…
We propose two numerical schemes for approximating quasi-stationary distributions (QSD) of finite state Markov chains with absorbing states. Both schemes are described in terms of certain interacting chains in which the interaction is given…
Representations based on random walks can exploit discrete data distributions for clustering and classification. We extend such representations from discrete to continuous distributions. Transition probabilities are now calculated using a…
Markov-modulated fluid queues are popular stochastic processes frequently used for modelling real-life applications. An important performance measure to evaluate in these applications is their steady-state behaviour, which is determined by…
In this paper, we present an overview of different types of random walk strategies with local and non-local transitions on undirected connected networks. We present a general approach to analyzing these strategies by defining the dynamics…
This article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30,197-207, (1986)…
Motivated by queues with many servers, we study Brownian steady-state approximations for continuous time Markov chains (CTMCs). Our approximations are based on diffusion models (rather than a diffusion limit) whose steady-state, we prove,…
In this paper we study the statistical properties of a reversible cellular automaton in two out-of-equilibrium settings. In the first part we consider two instances of the initial value problem, corresponding to the inhomogeneous quench and…
Nonequilibrium response theory is a fundamental framework for understanding how physical systems respond to perturbations. Recently, a mutual linearity has been discovered for Markov jump processes using linear algebra analysis. This mutual…
We study a discrete time queueing system where deterministic arrivals have i.i.d. exponential delays $\xi_{i}$. The standard deviation $\sigma$ of the delay is finite, but its value is much larger than the deterministic unit service time.…