Related papers: Admission Control of Quasi-Reversible Queueing Sys…
Completeness of a dynamic priority scheduling scheme is of fundamental importance for the optimal control of queues in areas as diverse as computer communications, communication networks, supply chains and manufacturing systems. Our first…
In this paper, two Q-learning (QL) methods are proposed and their convergence theories are established for addressing the model-free optimal control problem of general nonlinear continuous-time systems. By introducing the Q-function for…
This paper studies a scheduling control problem for a single-server multiclass queueing network in heavy traffic, operating in a changing environment. The changing environment is modeled as a finite state Markov process that modulates the…
We consider a model for transitory queues in which only a finite number of customers can join. The queue thus operates over a finite time horizon. In this system, also known as the $\Delta_{(i)}/G/1$ queue, the customers decide…
We propose two new optimistic planning algorithms for nonlinear hybrid-input systems, in which the input has both a continuous and a discrete component, and the discrete component must respect a dwell-time constraint. Both algorithms select…
We consider queueing systems with n parallel queues under a Join the Shortest Queue (JSQ) policy in the Halfin-Whitt heavy traffic regime. We use the martingale method to prove that a scaled process counting the number of idle servers and…
Motivated by health care systems with repeated services that have both personnel (nurse and physician) and space (beds) constraints, we study a restricted version of the Erlang-R model. The space restriction policies we consider are…
Model predictive control can optimally deal with nonlinear systems under consideration of constraints. The control performance depends on the model accuracy and the prediction horizon. Recent advances propose to use reinforcement learning…
We study flow shop scheduling with stochastic reentry, where jobs must complete multiple passes through the entire shop, and the number of passes that a job requires for completion is drawn from a discrete probability distribution. The goal…
This paper proposes a totally asymmetric simple exclusion process on a traveling lane, which is equipped with a queueing system and functions of site assignments along the parking lane. In the proposed system, new particles arrive at the…
The problem of designing a rebalancing algorithm for a large-scale ridehailing system with asymmetric demand is considered here. We pose the rebalancing problem within a semi Markov decision problem (SMDP) framework with closed queues of…
In asymptotic regimes, both in time and space (network size), the derivation of network capacity results is grossly simplified by brushing aside queueing behavior in non-Jackson networks. This simplifying double-limit model, however, lends…
The residual queue during a given study period (e.g., peak hour) is an important feature that should be considered when solving a traffic assignment problem under equilibrium for strategic traffic planning. Although studies have focused…
This brief proposes a quasi time-fuel optimal control strategy to solve the dynamic tracking problem of unmanned systems when fuel and control input are limited. This kind of motion planning and control strategy could bring the biggest…
We study real-time routing policies in smart transit systems, where the platform has a combination of cars and high-capacity vehicles (e.g., buses or shuttles) and seeks to serve a set of incoming trip requests. The platform can use its…
The expected regret of any reinforcement learning algorithm is lower bounded by $\Omega\left(\sqrt{DXAT}\right)$ for undiscounted returns, where $D$ is the diameter of the Markov decision process, $X$ the size of the state space, $A$ the…
We consider a system consisting of a single transmitter/receiver pair and $N$ channels over which they may communicate. Packets randomly arrive to the transmitter's queue and wait to be successfully sent to the receiver. The transmitter may…
We study $n$ parallel queues in an extreme heavy-traffic regime: each server works at rate $n$, while jobs arrive to a dispatcher at rate $n^2-(a-b)\sqrt{n}$, with fixed $a>b>0$. Arrivals are routed by a marginal join-the-shortest-queue…
The Join-the-Shortest-Queue (JSQ) policy is among the most widely used load balancing algorithms and has been extensively studied. However, an exact characterization of the system behavior remains challenging. Most prior research has…
The operating point of a power system may change due to slow enough variations of the power injections. Rotating machines in the bulk system can absorb smooth changes in the dynamic states of the system. In this context, we present a novel…