Related papers: SPLIT: SymPathy for Large jobs Improves Tail laten…
We provide upper bounds on the end-to-end backlog and delay in a network with heavy-tailed and self-similar traffic. The analysis follows a network calculus approach where traffic is characterized by envelope functions and service is…
We have been witnessing the usefulness of conversational AI systems such as Siri and Alexa, directly impacting our daily lives. These systems normally rely on machine learning models evolving over time to provide quality user experience.…
Heavy-tailed distributions are frequently used to enhance the robustness of regression and classification methods to outliers in output space. Often, however, we are confronted with "outliers" in input space, which are isolated observations…
The rigid gang task model is based on the idea of executing multiple threads simultaneously on a fixed number of processors to increase efficiency and performance. Although there is extensive literature on global rigid gang scheduling,…
In this paper, we study the asymptotic behavior of the tail probability of the number of customers in the steady-state $M/G/1$ retrial queue with Bernoulli schedule, under the assumption that the service time distribution has a regularly…
We study tandem queueing systems in which servers work more efficiently in teams than on their own and customers are impatient in that they may leave the system while waiting for service. Our goal is to determine the server assignment…
Tail asymptotics of the solution $R$ to a fixpoint problem of type $R =_{st} Q + \sum_1^N R_m$ is derived under heavy-tailed conditions allowing both dependence between $Q$ and $N$ and the tails to be of the same order of magnitude. Similar…
A multi-class single-server queueing model with finite buffers, in which scheduling and admission of customers are subject to control, is studied in the moderate deviation heavy traffic regime. A risk-sensitive cost set over a finite time…
Using data from cardiovascular surgery patients with long and highly variable post-surgical lengths of stay (LOS), we develop a modeling framework to reduce recovery unit congestion. We estimate the LOS and its probability distribution…
We consider large-scale service systems with multiple customer classes and multiple server pools; interarrival and service times are exponentially distributed, and mean service times depend both on the customer class and server pool. It is…
Given a finite collection of stochastic alternatives, we study the problem of sequentially allocating a fixed sampling budget to identify the optimal alternative with a high probability, where the optimal alternative is defined as the one…
Advances in Large Language Models (LLMs) have led to a surge of LLM-powered applications. These applications have diverse token-generation latency requirements. As a result, simply classifying workloads as latency-sensitive (LS) or…
The problem of online scheduling of multi-server jobs is considered, where there are a total of $K$ servers, and each job requires concurrent service from multiple servers for it to be processed. Each job on its arrival reveals its…
Efficient scheduling is crucial for interactive Large Language Model (LLM) applications, where low request completion time directly impacts user engagement. Size-based scheduling algorithms like Shortest Remaining Process Time (SRPT) aim to…
Hosting diverse large language model workloads in a unified resource pool through co-location is cost-effective. For example, long-running chat services generally follow diurnal traffic patterns, which inspire co-location of batch jobs to…
Querying graph data with low latency is an important requirement in application domains such as social networks and knowledge graphs. Graph queries perform multiple hops between vertices. When data is partitioned and stored across multiple…
Sequential user behavior modeling plays a crucial role in online user-oriented services, such as product purchasing, news feed consumption, and online advertising. The performance of sequential modeling heavily depends on the scale and…
Modern data centers are tasked with processing heterogeneous workloads consisting of various classes of jobs. These classes differ in their arrival rates, size distributions, and job parallelizability. With respect to paralellizability,…
Multi-server jobs are imperative in modern computing clusters. A multi-server job has multiple task components and each of the task components is responsible for processing a specific size of workloads. Efficient online workload dispatching…
We consider a system of parallel queues where tasks are assigned (dispatched) to one of the available servers upon arrival. The dispatching decision is based on the full state information, i.e., on the sizes of the new and existing jobs. We…