Related papers: Whittle Index Policy for Multichannel Scheduling i…

We propose a new one-bit feedback scheme with scheduling decision based on the maximum expected weighted rate. We show the concavity of the $2$-user case and provide the optimal solution which achieves the maximum weighted rate of the…

We consider the problem of joint routing and scheduling in queueing networks, where the edge transmission costs are unknown. At each time-slot, the network controller receives noisy observations of transmission costs only for those edges it…

The interest to retrial queueing systems is due to their application to telephone systems. The paper studies multiserver retrial queueing systems with $n$ servers. Arrival process is a quite general point process. An arriving customer…

In multi-server queueing systems where there is no central queue holding all incoming jobs, job dispatching policies are used to assign incoming jobs to the queue at one of the servers. Classic job dispatching policies such as…

We study the Whittle index learning algorithm for restless multi-armed bandits. We consider index learning algorithm with Q-learning. We first present Q-learning algorithm with exploration policies -- epsilon-greedy, softmax,…

With the growing density of wireless networks and demand for multi-hop transmissions, precise delay Quality of Service (QoS) analysis has become a critical challenge. This paper introduces a multi-hop delay QoS analysis framework based on…

Restless multi-armed bandits (RMABs) have been widely utilized to address resource allocation problems with Markov reward processes (MRPs). Existing works often assume that the dynamics of MRPs are known prior, which makes the RMAB problem…

In heterogeneous networks (HetNets), load balancing among different tiers can be effectively achieved by a biased user association scheme with which each user chooses to associate with one base station (BS) based on the biased received…

We present BLINQ, a new model-based algorithm that learns the Whittle indices of an indexable, communicating and unichain Markov Decision Process (MDP). Our approach relies on building an empirical estimate of the MDP and then computing its…

A system manager makes dynamic pricing and dispatch control decisions in a queueing network model motivated by ride-hailing applications. A novel feature of the model is that it incorporates travel times. Unfortunately, this renders the…

In many public health settings, it is important for patients to adhere to health programs, such as taking medications and periodic health checks. Unfortunately, beneficiaries may gradually disengage from such programs, which is detrimental…

A customary solution to reduce the energy consumption of wireless communication devices is to periodically put the radio into low-power sleep mode. A relevant problem is to schedule the wake-up of nodes in such a way as to ensure proper…

This report considers a fairly general model of constrained queuing networks that allows us to represent both MMBP (Markov Modulated Bernoulli Processes) arrivals and time-varying service constraints. We derive a set of sufficient…

Performance modeling is a key issue in queuing theory and operation research. It is well-known that the length of a queue that awaits service or the time spent by a job in a queue depends not only on the service rate, but also crucially on…

In this paper, we study a queueing model that incorporates patient reentrance to reflect patients' recurring requests for nurse care and their rest periods between these requests. Within this framework, we address two levels of…

We consider a wireless system with a small number of delay constrained users and a larger number of users without delay constraints. We develop a scheduling algorithm that reacts to time varying channels and maximizes throughput utility (to…

We develop link rate control policies to minimize the queueing delay of packets in overloaded networks. We show that increasing link rates does not guarantee delay reduction during overload. We consider a fluid queueing model that…

The dynamic allocation problem, also known as the `multi-armed bandit' problem, simulates a situation in which an agent is faced with a tradeoff between actions that yield an immediate reward and actions whose benefits can only be perceived…

We study the problem of serving randomly arriving and delay-sensitive traffic over a multi-channel communication system with time-varying channel states and unknown statistics. This problem deviates from the classical…

In this paper we develop scheduling policies that maximize the stability region of a wireless network under the assumption that mutual information accumulation is implemented at the physical layer. When the link quality between nodes is not…