Related papers: The Gittins Policy in the M/G/1 Queue
We develop asymptotically optimal policies for the multi armed bandit (MAB), problem, under a cost constraint. This model is applicable in situations where each sample (or activation) from a population (bandit) incurs a known bandit…
In multi-server queueing systems where there is no central queue holding all incoming jobs, job dispatching policies are used to assign incoming jobs to the queue at one of the servers. Classic job dispatching policies such as…
Generative models, like flows and diffusions, have recently emerged as popular and efficacious policy parameterizations in robotics. There has been much speculation as to the factors underlying their successes, ranging from capturing…
The pivot algorithm -- we also call it the pivot chain -- is an algorithm for approximately uniform sampling from $\Omega _{N},$ the set of $N$-step self-avoiding walks on $\mathbb{Z}^{d}$ ($N,$ $d\geq 1$). Based on this algorithm and the…
Solving planning and scheduling problems for multiple tasks with highly coupled state and temporal constraints is notoriously challenging. An appealing approach to effectively decouple the problem is to judiciously order the events such…
A multi-class single-server system with general service time distributions is studied in a moderate deviation heavy traffic regime. In the scaling limit, an optimal control problem associated with the model is shown to be governed by a…
In this paper we consider a real time queuing system with rewards and deadlines. We assume that packet processing time is known upon arrival, as is the case in communication networks. This assumption allows us to demonstrate that the well…
Scheduling precedence-constrained tasks is a classical problem that has been studied for more than fifty years. However, little progress has been made in the setting where there are communication delays between tasks. Results for the case…
A restless multi-armed bandit problem that arises in multichannel opportunistic communications is considered, where channels are modeled as independent and identical Gilbert-Elliot channels and channel state observations are subject to…
Motivated by practical applications, chiefly clinical trials, we study the regret achievable for stochastic bandits under the constraint that the employed policy must split trials into a small number of batches. We propose a simple policy,…
In stochastic bandit problems, a Bayesian policy called Thompson sampling (TS) has recently attracted much attention for its excellent empirical performance. However, the theoretical analysis of this policy is difficult and its asymptotic…
Considering a lead-time-and price-sensitive demand, we investigate whether a client rejection policy, modeled as M/M/1/K system, can be more profitable than an all-client acceptance policy, modeled as M/M/1 system. We provide analytical…
Consider a multi-phase project management problem where the decision maker needs to deal with two issues: (a) how to allocate resources to projects within each phase, and (b) when to enter the next phase, so that the total expected reward…
In 2022, Chen et al. proposed an algorithm in \cite{main} that solves the min cost flow problem in $m^{1 + o(1)} \log U \log C$ time, where $m$ is the number of edges in the graph, $U$ is an upper bound on capacities and $C$ is an upper…
We study the effect of restart, and retry, on the mean completion time of a generic process. The need to do so arises in various branches of the sciences and we show that it can naturally be addressed by taking advantage of the classical…
Service systems like data centers and ride-hailing are popularly modeled as queueing systems in the literature. Such systems are primarily studied in the steady state due to their analytical tractability. However, almost all applications in…
Multiserver queueing systems are found at the core of a wide variety of practical systems. Many important multiserver models have a previously-unexplained similarity: identical mean response time behavior is empirically observed in the…
Decision-making problems of sequential nature, where decisions made in the past may have an impact on the future, are used to model many practically important applications. In some real-world applications, feedback about a decision is…
For traffic routing platforms, the choice of which route to recommend to a user depends on the congestion on these routes -- indeed, an individual's utility depends on the number of people using the recommended route at that instance.…
The Whittle index, which characterizes optimal policies for controlling certain single restless bandit projects (a Markov decision process with two actions: active and passive) is the basis for a widely used heuristic index policy for the…