Related papers: Optimal control of Markov-modulated multiclass man…
In this paper we analyze an $M/M/1$ queueing system with an arbitrary number of customer classes, with class-dependent exponential service rates and preemptive priorities between classes. The queuing system can be described by a…
This paper develops a quantized Q-learning algorithm for the optimal control of controlled diffusion processes on $\mathbb{R}^d$ under both discounted and ergodic (average) cost criteria. We first establish near-optimality of finite-state…
In this paper we study a non-stationary Markovian queueing model of a two-processor heterogeneous system with time-varying arrival and service rates. We obtain the bounds on the rate of convergence and find the main limiting characteristics…
We study a double-ended queue which consists of two classes of customers. Whenever there is a pair of customers from both classes, they are matched and leave the system immediately. The matching follows first-come-first-serve principle. If…
In this work, we consider a continuous-time inventory system where the demand process follows an inventory-dependent diffusion process. The ordering cost of each order depends on the order quantity and is given by a general function, which…
We study a general $k$ dimensional infinite server queues process with Markov switching, Poisson arrivals and where the service times are fat tailed with index $\alpha\in (0,1)$. When the arrival rate is sped up by a factor $n^\gamma$, the…
We consider exponential single server queues with state-dependent arrival and service rates which evolve under influences of external environments. The transitions of the queues are influenced by the environment's state and the movements of…
We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite…
In queuing theory and related problems, it is very important to know the numerical characteristics of an investigated system - both in stationary and non-stationary modes. In some cases, such characteristics can be calculated, but this is…
Service systems like data centers and ride-hailing are popularly modeled as queueing systems in the literature. Such systems are primarily studied in the steady state due to their analytical tractability. However, almost all applications in…
We study a two-stage tandem service queue attended by two servers. Each job-server pair must complete both service phases together, with the server unable to begin a new job until the current one is fully processed after two stages.…
We propose a unified approach to establishing diffusion approximations for queues with impatient customers within a general framework of scaling customer patience time. The approach consists of two steps. The first step is to show that the…
We consider a stationary Markov process that models certain queues with a bulk service of a fixed number $m$ of admitted customers. We find an integral expression of its transition probability function in terms of certain multi-orthogonal…
Stochastic optimal control problems have a long tradition in applied probability, with the questions addressed being of high relevance in a multitude of fields. Even though theoretical solutions are well understood in many scenarios, their…
This paper introduces and analyzes the notion of throughput suboptimality for many-server queueing systems in heavy traffic. The queueing model under consideration has multiple customer classes, indexed by a finite set $\mathcal{I}$, and…
In previous papers we developed a deterministic fluid approximation for an overloaded Markovian queueing system having two customer classes and two service pools, known in the call-center literature as the X model. The system uses the…
In parallel-server systems with a single stream of arrivals (a.k.a. load balancing), Join-the-Shortest-Queue (JSQ) is a popular routing algorithm. There is extensive literature studying this system in various asymptotic regimes, but all…
In this paper we revisit the Markovian queueing system with a single server, infinite capacity queue and the special queue skipping policy. Customers arrive in batches, but are served one by one according to any conservative discipline. The…
This paper addresses the analysis of the queue-length process of single-server queues under overdispersion, i.e., queues fed by an arrival process for which the variance of the number of arrivals in a given time window exceeds the…
In this paper, we establish a general stochastic maximum principle for optimal control for systems described by a continuous-time Markov regime-switching stochastic recursive utilities model. The control domain is postulated not to be…