Related papers: Bounds on the rate on convergence for Markovian qu…
In this paper, we consider queueing systems where the dynamics are non-stationary and state-dependent. For performance analysis of these systems, fluid and diffusion models have been typically used. Although they are proven to be…
A general formalism is developed to construct a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are therefore internal to the system and not externally specified. For…
The paper presents efficient approaches for evaluating convergence rate in total variation for finite and general linear Markov chains. The motivation for studying convergence rate in this metric is its usefulness in various limit theorems.…
This paper considers stochastic-constrained stochastic optimization where the stochastic constraint is to satisfy that the expectation of a random function is below a certain threshold. In particular, we study the setting where data samples…
We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…
Lyapunov drift and Lyapunov optimization are powerful techniques for optimizing time averages in stochastic queueing networks subject to stability. However, there are various definitions of queue stability in the literature, and the most…
We consider the well-known problem of the computation of the (limiting) time-dependent performance characteristics of one-dimensional continuous-time birth and death processes on $\mathbb{Z}$ with time varying and possible state-dependent…
Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend…
We study continuous-time Markov chains on the non-negative integers under mild regularity conditions (in particular, the set of jump vectors is finite and both forward and backward jumps are possible). Based on the so-called flux balance…
Consider a system performing a continuous-time random walk on the integers, subject to catastrophes occurring at constant rate, and followed by exponentially-distributed repair times. After any repair the system starts anew from state zero.…
We study the problem of learning the Markov order in categorical sequences that represent paths in a network, i.e. sequences of variable lengths where transitions between states are constrained to a known graph. Such data pose challenges…
Testing for regime switching when the regime switching probabilities are specified either as constants (`mixture models') or are governed by a finite-state Markov chain (`Markov switching models') are long-standing problems that have also…
In this paper some general theory is presented for locally stationary processes based on the stationary approximation and the stationary derivative. Laws of large numbers, central limit theorems as well as deterministic and stochastic bias…
There are multiple ways in which a stochastic system can be out of statistical equilibrium. It might be subject to time-varying forcing; or be in a transient phase on its way towards equilibrium; it might even be in equilibrium without us…
We propose a model for epidemic spreading on a finite complex network with a restriction to at most one contamination per time step. Because of a highly discrete character of the process, the analysis cannot use the continous approximation,…
We study normal approximations for a class of discrete-time occupancy processes, namely, Markov chains with transition kernels of product Bernoulli form. This class encompasses numerous models which appear in the complex networks…
We provide a sharp nonasymptotic analysis of the rates of convergence for some standard multivariate Markov chains using spectral techniques. All chains under consideration have multivariate orthogonal polynomial as eigenfunctions. Our…
We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of…
We consider the averaging principle for deterministic or stochastic systems with a fast stochastic component (family of continuous-time Markov chains depending on the state of the system as a parameter). We show that, due to bifurcations in…
We consider Bayesian hierarchical models for survival analysis, where the survival times are modeled through an underlying diffusion process which determines the hazard rate. We show how these models can be efficiently treated by means of…