Related papers: Load is not what you should balance: Introducing P…
Disaggregated inference has become an essential framework that separates the prefill (P) and decode (D) stages in large language model inference to improve throughput. However, the KV cache transfer faces significant delays between prefill…
Content Delivery Networks carry the majority of Internet traffic, and the increasing demand for video content as a major IP traffic across the Internet highlights the importance of caching and prefetching optimization algorithms.…
Streaming video models should respond the moment an event unfolds, not after the moment has passed. Yet existing online VideoQA benchmarks remain largely retrospective. They pause the video at fixed timestamps, pose questions about current…
Decentralized proactive caching and coded delivery is studied in a content delivery network, where each user is equipped with a cache memory, not necessarily of equal capacity. Cache memories are filled in advance during the off-peak…
A load balancer (LB) is a vital network function for cloud services to balance the load amongst resources. Stateful software LBs that run on commodity servers provides flexibility, cost-efficiency, and packet consistency. However current…
Use-case-specific network slicing in decentralized multi-tenancy cloud environments is a promising approach to bridge the gap between the demand and supply of resources in next-generation communication networks. Our findings associate…
Minimizing waiting time for tasks waiting in the queue for execution is one of the important scheduling cri-teria which took a wide area in scheduling preemptive tasks. In this paper we present Changeable Time Quan-tum (CTQ) approach…
In cloud computing environment, load balancing is a key issue which is required to distribute the dynamic workload over multiple machines to make certain that no single machine is overloaded. In recent research, many organizations lose…
Public clouds increasingly expose heterogeneous hardware, but their allocation interface remains built around rigid on-demand and spot service classes. This makes it hard to satisfy time-varying tenant objectives and operator constraints in…
The back-pressure algorithm is a well-known throughput-optimal algorithm. However, its delay performance may be quite poor even when the traffic load is not close to network capacity due to the following two reasons. First, each node has to…
Nowadays, more and more increasingly hard computations are performed in challenging fields like weather forecasting, oil and gas exploration, and cryptanalysis. Many of such computations can be implemented using a computer cluster with a…
MapReduce framework is the de facto in big data and its applications where a big data-set is split into small data chunks that are replicated on different servers among thousands of servers. The heterogeneous server structure of the system…
We present a probabilistic proactive rebalancing method and speed-up techniques for improving the performance of a state-of-the-art real-time high-capacity fleet management framework [1]. We improve on both computational efficiency and…
Efficient virtual machine load balancing scheduling is crucial in cloud computing to optimize resource utilization and system performance. To address this issue, several load balancing scheduling algorithms have been proposed, including…
This work considers a parallel task execution strategy in vehicular edge computing (VEC) networks, where edge servers are deployed along the roadside to process offloaded computational tasks of vehicular users. To minimize the overall…
Cloud systems have rapidly expanded worldwide in the last decade, shifting computational tasks to cloud servers where clients submit their requests. Among cloud workloads, latency-critical applications -- characterized by high-percentile…
Distributed clusters like the Grid and PlanetLab enable the same statistical multiplexing efficiency gains for computing as the Internet provides for networking. One major challenge is allocating resources in an economically efficient and…
Monolithic serving with chunked prefill improves GPU utilization by batching prefill and decode together, but suffers from fine-grained phase interference. Engine-level prefill-decode (PD) disaggregation avoids interference but incurs…
Efficiently exploiting the resources of data centers is a complex task that requires efficient and reliable load balancing and resource allocation algorithms. The former are in charge of assigning jobs to servers upon their arrival in the…
In this paper, we investigate a multi-user offloading problem in the overlapping domain of a multi-server mobile edge computing system. We divide the original problem into two stages: the offloading decision making stage and the request…