Related papers: Load is not what you should balance: Introducing P…
Large Language Models (LLMs) rely on optimizations like Automatic Prefix Caching (APC) to accelerate inference. APC works by reusing previously computed states for the beginning part of a request (prefix), when another request starts with…
We consider ML query processing in distributed systems where GPU-enabled workers coordinate to execute complex queries: a computing style often seen in applications that interact with users in support of image processing and natural…
Modern LLM serving systems confront inefficient GPU utilization due to the fundamental mismatch between compute-intensive prefill and memory-bound decode phases. While current practices attempt to address this by organizing these phases…
Meta computing is a new computing paradigm that aims to efficiently utilize all network computing resources to provide fault-tolerant, personalized services with strong security and privacy guarantees. It also seeks to virtualize the…
In today's private cloud, the resource of the datacenter is shared by multiple tenants. Unlike the storage and computing resources, it's challenging to allocate bandwidth resources among tenants in private datacenter networks.…
With the rising number of distributed computer systems, from microservice web applications to IoT platforms, the question of reliable communication between different parts of the aforementioned systems is becoming increasingly important. As…
Cloud computing has grown rapidly in recent years, mainly due to the sharp increase in data transferred over the internet. This growth makes load balancing a key part of cloud systems, as it helps distribute user requests across servers to…
Recent exponential growth of data over cellular networks has cause the progression from conventional mobile communication networks to heterogeneous cellular networks (HetNets). Quality of experience (QoE)-aware traffic load balancing in…
Load balancing plays a crucial role in many large scale systems. Several different load balancing principles have been proposed in the literature, such as Join-Shortest-Queue (JSQ) and its variations, or Join-Below-Threshold. We provide a…
The cloud datacenter has numerous hosts as well as application requests where resources are dynamic. The demands placed on the resource allocation are diverse. These factors could lead to load imbalances, which affect scheduling efficiency…
Modern deployments of Large Language Models (LLMs) increasingly require serving multiple models with diverse architectures, sizes, and specialization on shared, heterogeneous hardware. This setting introduces new challenges for resource…
Prompts to large language models (LLMs) have evolved beyond simple user questions. For LLMs to solve complex problems, today's practices are to include domain-specific instructions, illustration of tool usages, and/or long context such as…
In large-scale ranking systems, cascading architectures have been widely adopted to achieve a balance between efficiency and effectiveness. The pre-ranking module plays a vital role in selecting a subset of candidates for the subsequent…
As computation shifts from the cloud to the edge to reduce processing latency and network traffic, the resulting Computing Continuum (CC) creates a dynamic environment where meeting strict Quality of Service (QoS) requirements and avoiding…
Network virtualization and SDN-based routing allow carriers to flexibly configure their networks in response to demand and unexpected network disruptions. However, cellular networks, by nature, pose some unique challenges because of user…
The ever-increasing demand for data services and the proliferation of user equipment (UE) have resulted in a significant rise in the volume of mobile traffic. Moreover, in multi-band networks, non-uniform traffic distribution among…
In order to utilize network resources and provide a more reliable delivery of information, multipath routing algorithms are used with a higher priority than single path routing. But providing the quality of service (QoS) requirements while…
Low delay is an explicit requirement for applications such as cloud gaming and video conferencing. Delay-based congestion control can achieve the same throughput but significantly smaller delay than loss-based one and is thus ideal for…
We propose a unified coded framework for distributed computing with straggling servers, by introducing a tradeoff between "latency of computation" and "load of communication" for some linear computation tasks. We show that the coded scheme…
On-Demand Ride-Pooling services have the potential to increase traffic efficiency compared to private vehicle trips by decreasing parking space needed and increasing vehicle occupancy due to higher vehicle utilization and shared trips,…