Related papers: Stationary Markovian Arrival Processes, Results an…
We consider qualitative and quantitative verification problems for infinite-state Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which…
The mean first-passage time (MFPT) is one standard measure for the reaction time in thermally activated barrier-crossing processes. While the relationship between MFPTs and phenomenological rate coefficients is known for systems that…
In this paper, we study Markov-modulated dependencies for the multiplicative Lindley's recursion $W_{n+1}=[V_{n}W_{n}+Y_{n}(V_{n})]^{+}$, where $Y_{n}(V_{n})$ may depend on $V_{n}$, and can be written as the difference of two nonnegative…
An important fundamental problem in green communications and networking is the operation of servers (routers or base stations) with sleeping mechanism to optimize energy-delay tradeoffs. This problem is very challenging when considering…
In this paper, we consider semi-Markov processes whose transition times and transition probabilities depend on a small parameter $\varepsilon$. Understanding the asymptotic behavior of such processes is needed in order to study the…
We give a complete classification of integrable Markovian boundary conditions for the asymmetric simple exclusion process with two species (or classes) of particles. Some of these boundary conditions lead to non-vanishing particle currents…
We continue the investigation of the spectral theory and exponential asymptotics of Markov processes, following Kontoyiannis and Meyn (2003). We introduce a new family of nonlinear Lyapunov drift criteria, characterizing distinct subclasses…
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary…
Markov decision processes (MDPs) are a fundamental model for decision making under uncertainty. They exhibit non-deterministic choice as well as probabilistic uncertainty. Traditionally, verification algorithms assume exact knowledge of the…
Markov state models (MSMs) have been widely used to analyze computer simulations of various biomolecular systems. They can capture conformational transitions much slower than an average or maximal length of a single molecular dynamics (MD)…
We define dynamic treatment regimes and associated potential outcomes for data described by marked point processes (MPPs). These definitions motivate MPP analogues of the commonly used consistency, exchangeability, and positivity conditions…
We consider partially observable Markov decision processes (POMDPs) with limit-average payoff, where a reward value in the interval [0,1] is associated to every transition, and the payoff of an infinite path is the long-run average of the…
We consider a Markovian jumping process with two absorbing barriers, for which the waiting-time distribution involves a position-dependent coefficient. We solve the Fokker-Planck equation with boundary conditions and calculate the mean…
We introduce the notion of order of magnitude reversibility (OM-reversibility) in Markov chains that are parametrized by a positive parameter $\ep$. OM-reversibility is a weaker condition than reversibility, and requires only the knowledge…
A determinantal point process (DPP) is an ensemble of random nonnegative-integer-valued Radon measures, whose correlation functions are all given by determinants specified by an integral kernel called the correlation kernel. First we show…
In this work, we present a general method to establish properties of multi-dimensional continuous-time Markov chains representing stochastic reaction networks. This method consists of grouping states together (via a partition of the state…
This paper presents an identity between the multivariate and univariate saddlepoint approximations applied to sample path probabilities for a certain class of stochastic processes. This class, which we term the recursively compounded…
We consider partially observable Markov decision processes (POMDPs) with limit-average payoff, where a reward value in the interval [0,1] is associated to every transition, and the payoff of an infinite path is the long-run average of the…
Let $P$ be the transition matrix of a finite, irreducible and reversible Markov chain. We say the continuous time Markov chain $X$ has transition matrix $P$ and speed $\lambda$ if it jumps at rate $\lambda$ according to the matrix $P$. Fix…
The random numbers driving Markov chain Monte Carlo (MCMC) simulation are usually modeled as independent U(0,1) random variables. Tribble [Markov chain Monte Carlo algorithms using completely uniformly distributed driving sequences (2007)…