Related papers: SPLIT: SymPathy for Large jobs Improves Tail laten…
We investigate the tail asymptotics of the response time distribution for the cancel-on-start (c.o.s.) and cancel-on-completion (c.o.c.) variants of redundancy-$d$ scheduling and the fork-join model with heavy-tailed job sizes. We present…
We consider a computation offloading system where jobs are processed sequentially at a local server followed by a higher-capacity cloud server. The system offers two service modes, differing in how the processing is split between the…
Efficient data access in High-Performance Computing (HPC) systems is essential to the performance of intensive computing tasks. Traditional optimizations of the I/O stack aim to improve peak performance but are often workload specific and…
We say that a random variable is $light$-$tailed$ if moments of order $2+\epsilon$ are finite for some $\epsilon>0$; otherwise, we say that it is $heavy$-$tailed$. We study queueing networks that operate under the Max-Weight scheduling…
We consider the single server queue with service in random order. For a large class of heavy-tailed service time distributions, we determine the asymptotic behavior of the waiting time distribution. For the special case of Poisson arrivals…
Can the popular shortest remaining processing time (SRPT) algorithm achieve a constant competitive ratio on multiple servers when server speeds are adjustable (speed scaling) with respect to the flow time plus energy consumption metric?…
In cloud computing systems, assigning a task to multiple servers and waiting for the earliest copy to finish is an effective method to combat the variability in response time of individual servers, and reduce latency. But adding redundancy…
We investigate the application of the Adaptive Multilevel Splitting algorithm for the estimation of tail probabilities of solutions of Stochastic Differential Equations evaluated at a given time, and of associated temporal averages. We…
State-machine replication, a fundamental approach to fault tolerance, requires replicas to execute commands deterministically, which usually results in sequential execution of commands. Sequential execution limits performance and underuses…
Serving systems for Large Language Models (LLMs) improve throughput by processing several requests concurrently. However, multiplexing hardware resources between concurrent requests involves non-trivial scheduling decisions. Practical…
Real-world visual data often exhibits a long-tailed distribution, where some ''head'' classes have a large number of samples, yet only a few samples are available for ''tail'' classes. Such imbalanced distribution causes a great challenge…
We consider maximizing the long-term average reward in a single server queue, where the reward obtained for a job is a non-increasing function of its sojourn time. The motivation behind this work comes from multiple applications, including…
We examine a distributional fixed-point equation related to a multi-type branching process that is key in the cluster sizes analysis of multivariate heavy-tailed Hawkes processes. Specifically, we explore the tail behavior of its solution…
We develop accurate approximations of the delay distribution of the MArP/G/1 queue that cap- ture the exact tail behavior and provide bounded relative errors. Motivated by statistical analysis, we consider the service times as a mixture of…
Self-improvement methods enable large language models (LLMs) to generate solutions themselves and iteratively train on filtered, high-quality rationales. This process proves effective and reduces the reliance on human supervision in LLMs'…
Today's search engines process billions of online user queries a day over huge collections of data. In order to scale, they distribute query processing among many nodes, where each node holds and searches over a subset of the index called…
Modern datacenter applications are prone to high tail latencies since their requests typically follow highly-dispersive distributions. Delivering fast interrupts is essential to reducing tail latency. Prior work has proposed both OS- and…
Large language model (LLM) serving is becoming an increasingly critical workload for cloud providers. Existing LLM serving systems focus on interactive requests, such as chatbots and coding assistants, with tight latency SLO requirements.…
We study the problem of scheduling jobs on fault-prone machines communicating via a shared channel, also known as multiple-access channel. We have $n$ arbitrary length jobs to be scheduled on $m$ identical machines, $f$ of which are prone…
We address the problem of giving robust performance bounds based on the study of the asymptotic behavior of the insensitive load balancing schemes when the number of servers and the load scales jointly. These schemes have the desirable…