Related papers: Tail Optimality and Performance Analysis of the Nu…
Large Language Models (LLMs) represent a revolutionary advancement in the contemporary landscape of artificial general intelligence (AGI). As exemplified by ChatGPT, LLM-based applications necessitate minimal response latency and maximal…
We obtain asymptotic bounds for the tail distribution of steady-state waiting time in a two server queue where each server processes incoming jobs at a rate equal to the rate of their arrivals (that is, the half-loaded regime). The job…
We investigate the asymptotic behavior of the steady-state queue length distribution under generalized max-weight scheduling in the presence of heavy-tailed traffic. We consider a system consisting of two parallel queues, served by a single…
The ever-increasing gap between compute and I/O performance in HPC platforms, together with the development of novel NVMe storage devices (NVRAM), led to the emergence of the burst buffer concept - an intermediate persistent storage layer…
Serving Large Language Models (LLMs) under mixed workloads--short, latency-sensitive interactive queries alongside long, throughput-oriented batch requests--poses a fundamental scheduling challenge. Standard First-Come, First-Served (FCFS)…
In this paper we study the maximum queue length $M$ (in terms of the number of customers present) in a busy cycle in the M/G/1 queue. Assume that the service times have a logconvex density. For such (heavy-tailed) service-time distributions…
Modern computing workloads are often composed of parallelizable jobs. A parallelizable job can be completed more quickly when run on additional servers. However, each job can only use a limited number of servers, known as its…
Heavy-tailed noise has attracted growing attention in nonconvex stochastic optimization, as numerous empirical studies suggest it offers a more realistic assumption than standard bounded variance assumption. In this work, we investigate…
Microservice-based applications are characterized by stochastic latencies arising from long-tail execution patterns and heterogeneous resource constraints across computational nodes. To address this challenge, we first formulate the problem…
We present upper and lower bounds for the tail distribution of the stationary waiting time $D$ in the stable $GI/GI/s$ FCFS queue. These bounds depend on the value of the traffic load $\rho$ which is the ratio of mean service and mean…
We consider the stochastic scheduling problem of minimizing the expected makespan on $m$ parallel identical machines. While the (adaptive) list scheduling policy achieves an approximation ratio of $2$, any (non-adaptive) fixed assignment…
In this paper, we have proposed a new variant of Round Robin scheduling algorithm by executing the processes according to the new calculated Fit Factor f and using the concept of dynamic time quantum. We have compared the performance of our…
The ever-increasing gap between compute and I/O performance in HPC platforms, together with the development of novel NVMe storage devices (NVRAM), led to the emergence of the burst buffer concept - an intermediate persistent storage layer…
Existing research on single-machine scheduling is largely focused on exact algorithms, which perform well on typical instances but can significantly deteriorate on certain regions of the problem space. In contrast, data-driven approaches…
We consider the problem of preemptively scheduling jobs to minimize mean response time of an M/G/1 queue. When we know each job's size, the shortest remaining processing time (SRPT) policy is optimal. Unfortunately, in many settings we do…
In queueing systems, effective scheduling algorithms are essential for optimizing performance. Optimal scheduling for the M/G/k queue has been explored in the heavy traffic limit, but much remains unknown in the intermediate load regime. In…
This paper describes a novel method to solve average-reward semi-Markov decision processes, by reducing them to a minimal sequence of cumulative reward problems. The usual solution methods for this type of problems update the gain (optimal…
Large Language Models (LLMs) such as GPT-4 and Llama3 can already comprehend complex commands and process diverse tasks. This advancement facilitates their application in controlling drones and robots for various tasks. However, existing…
In this paper, we study the maximum waiting time $\max_{i\leq N}W_i(\cdot)$ in an $N$-server fork-join queue with heavy-tailed services as $N\to\infty$. The service times are the product of two random variables. One random variable has a…
Multiserver jobs, which are jobs that occupy multiple servers simultaneously during service, are prevalent in today's computing clusters. But little is known about the delay performance of systems with multiserver jobs. We consider queueing…