Related papers: Performance measures for the two-node queue with f…

This paper investigates a partially observable queueing system with $N$ nodes in which each node has a dedicated arrival stream. There is an extra arrival stream to balance the load of the system by routing its customers to the shortest…

In order to obtain Markov heavy-traffic approximations for infinite-server queues with general non-exponential service-time distributions and general arrival processes, possibly with time-varying arrival rates, we establish heavy-traffic…

We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…

We study the so-called two-time-scale stochastic approximation, a simulation-based approach for finding the roots of two coupled nonlinear operators. Our focus is to characterize its finite-time performance in a Markov setting, which often…

In this paper, we consider queueing systems where the dynamics are non-stationary and state-dependent. For performance analysis of these systems, fluid and diffusion models have been typically used. Although they are proven to be…

In this short communication we study a fluid queue with a finite buffer. The performance measure we are interested in is the occupation time over a finite time period, i.e., the fraction of time the workload process is below some fixed…

Computing the probability of evidence even with known error bounds is NP-hard. In this paper we address this hard problem by settling on an easier problem. We propose an approximation which provides high confidence lower bounds on…

We consider optimal control of a stochastic network,where service is controlled to prevent buffer overflow. We use a risk-sensitive escape time criterion, which in comparison to the ordinary escape time criteria heavily penalizes exits…

We consider a Markov chain on $\mathbb{R}^d$ with invariant measure $\mu$. We are interested in the rate of convergence of the empirical measures towards the invariant measure with respect to various dual distances, including in particular…

Supermarket models with different servers become a key in modeling resource management of stochastic networks, such as, computer networks, manufacturing systems and transportation networks. While these different servers always make analysis…

The random walk is a fundamental stochastic process that underlies many numerical tasks in scientific computing applications. We consider here two neural algorithms that can be used to efficiently implement random walks on spiking…

Queue networks describe complex stochastic systems of both theoretical and practical interest. They provide the means to assess alterations, diagnose poor performance and evaluate robustness across sets of interconnected resources. In the…

We consider conservative cross-diffusion systems for two species where individual motion rates depend linearly on the local density of the other species. We develop duality estimates and obtain stability and approximation results. We first…

We consider finite horizon Markov decision processes under performance measures that involve both the mean and the variance of the cumulative reward. We show that either randomized or history-based policies can improve performance. We prove…

In this paper, we study a controllable tandem queueing system consisting of two nodes and a controller, in which customers arrive according to a Poisson process and must receive service at both nodes before leaving the system. A decision…

We consider the batch (off-line) policy learning problem in the infinite horizon Markov Decision Process. Motivated by mobile health applications, we focus on learning a policy that maximizes the long-term average reward. We propose a…

Let $X$ be the constrained random walk on ${\mathbb Z}_+^d$ representing the queue lengths of a stable Jackson network and $x$ its initial position. Let $\tau_n$ be the first time the sum of the components of $X$ equals $n$. $p_n \doteq…

We study continuous-time Markov chains on the non-negative integers under mild regularity conditions (in particular, the set of jump vectors is finite and both forward and backward jumps are possible). Based on the so-called flux balance…

We propose a new Quantization algorithm for the approximation of inhomogeneous random walks, which are the key terms for the valuation of CDO-tranches in latent factor models. This approach is based on a dual quantization operator which…

We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…