Related papers: Tail Optimality and Performance Analysis of the Nu…
We propose ELIS, a serving system for Large Language Models (LLMs) featuring an Iterative Shortest Remaining Time First (ISRTF) scheduler designed to efficiently manage inference tasks with the shortest remaining tokens. Current LLM serving…
In this paper we describe a perfect simulation algorithm for the stable $M/G/c$ queue. Sigman (2011: Exact Simulation of the Stationary Distribution of the FIFO M/G/c Queue. Journal of Applied Probability, 48A, 209--213) showed how to build…
Stochastic networks with complex structures are key modelling tools for many important applications. In this paper, we consider a specific type of network: the retrial queueing systems with priority. This type of queueing system is…
Motivated by mail delivery scheduling problems arising in Royal Mail, we study a generalization of the fundamental makespan scheduling P||Cmax problem which we call the bounded job start scheduling problem. Given a set of jobs, each…
Two of the most popular approximations for the distribution of the steady-state waiting time, $W_{\infty}$, of the M/G/1 queue are the so-called heavy-traffic approximation and heavy-tailed asymptotic, respectively. If the traffic…
We consider a single large language model (LLM) server that serves a heterogeneous stream of queries belonging to $N$ distinct task types. Queries arrive according to a Poisson process, and each type occurs with a known prior probability.…
We study the problem of scheduling delay-sensitive jobs over spot and on-demand cloud instances to minimize average cost while meeting an average delay constraint. Jobs arrive as a general stochastic process, and incur different costs based…
Large Language Models have revolutionized natural language processing, yet serving them efficiently in data centers remains challenging due to mixed workloads comprising latency-sensitive (LS) and best-effort (BE) jobs. Existing inference…
This work studies the behavior of state-of-the-art memory controller designs when executing scale-out workloads. It considers memory scheduling techniques, memory page management policies, the number of memory channels, and the address…
We consider a distributed computing network consisting of a master and multiple workers processing tasks of different types. The master is running multiple applications. Each application stochastically generates real-time jobs with a strict…
In this paper we analyze the worst-case performance of a greedy algorithm called Largest-Z-ratio-First for the problem of scheduling unreliable jobs on m parallel machines. Each job is characterized by a success probability and a reward…
Inference-time alignment effectively steers large language models (LLMs) by generating multiple candidates from a reference model and selecting among them with an imperfect reward model. However, current strategies face a fundamental…
This paper presents a scalable solution with adjustable computation time for the joint problem of scheduling and assigning machines and transporters for missions that must be completed in a fixed order of operations across multiple stages.…
Interval scheduling is a basic problem in the theory of algorithms and a classical task in combinatorial optimization. We develop a set of techniques for partitioning and grouping jobs based on their starting and ending times, that enable…
Reinforcement Learning (RL) is a pivotal post-training technique for enhancing the reasoning capabilities of Large Language Models (LLMs). However, synchronous RL post-training often suffers from significant GPU underutilization, referred…
Network Function Virtualization (NFV) can cost-efficiently provide network services by running different virtual network functions (VNFs) at different virtual machines (VMs) in a correct order. This can result in strong couplings between…
In the moldable job scheduling problem one has to assign a set of $n$ jobs to $m$ machines, in order to minimize the time it takes to process all jobs. Each job is moldable, so it can be assigned not only to one but any number of the equal…
Our interest lies in load balancing jobs in large scale systems consisting of multiple dispatchers and FCFS servers. In the absence of any information on job sizes, dispatchers typically use queue length information reported by the servers…
In this paper, the asymptotic behaviour of the distribution tail of the stationary waiting time $W$ in the $GI/GI/2$ FCFS queue is studied. Under subexponential-type assumptions on the service time distribution, bounds and sharp asymptotics…
Space-time adaptive processing (STAP) is one of the most effective approaches to suppressing ground clutters in airborne radar systems. It basically takes two forms, i.e., full-dimension STAP (FD-STAP) and reduced-dimension STAP (RD-STAP).…