Related papers: Load is not what you should balance: Introducing P…
The ever-increasing demands of end-users on the Internet of Things (IoT), often cause great congestion in the nodes that serve their requests. Therefore, the problem of node overloading arises. In this article we attempt to solve the…
Load balancing is critical for distributed storage to meet strict service-level objectives (SLOs). It has been shown that a fast cache can guarantee load balancing for a clustered storage system. However, when the system scales out to…
The rapid proliferation of latency-sensitive and battery-constrained Internet-of-Things (IoT) applications has intensified the need for intelligent workload placement mechanisms across the Edge-Cloud computing continuum. In such…
Hardware prefetching plays a critical role in hiding the off-chip DRAM latency. The complexity of applications results in a wide variety of memory access patterns, prompting the development of numerous cache-prefetching algorithms.…
Computation offloading and resource allocation are critical in mobile edge computing (MEC) systems to handle the massive and complex requirements of applications restricted by limited resources. In a multi-user multi-server MEC network, the…
Today, large language models have demonstrated their strengths in various tasks ranging from reasoning, code generation, and complex problem solving. However, this advancement comes with a high computational cost and memory requirements,…
Queues with setup time are extensively studied because they have application in performance evaluation of power-saving data centers. In a data center, there are a huge number of servers which consume a large amount of energy. In the current…
As machine learning inferences increasingly move to edge devices, adapting to diverse computational capabilities, hardware, and memory constraints becomes more critical. Instead of relying on a pre-trained model fixed for all future…
Rapid advancements in cloud based platforms providing access to quantum computing capabilities have opened up several challenges for efficient usage of these highly delicate and costly devices. Although most of the current systems use a…
Federated learning is proposed by Google to safeguard data privacy through training models locally on users' devices. However, with deep learning models growing in size to achieve better results, it becomes increasingly difficult to…
Load-Balancers play an important role in data centers as they distribute network flows across application servers and guarantee per-connection consistency. It is hard however to make fair load balancing decisions so that all resources are…
Future network services present a significant challenge for network providers due to high number and high variety of co-existing requirements. Despite many advancements in network architectures and management schemes, congested network…
In the context of load balancing, Lu et al. introduced the distributed Join-Idle-Queue algorithm, where a group of dispatchers distribute jobs to a cluster of parallel servers. Each dispatcher maintains a queue of idle servers; when a job…
Efficiently harnessing GPU compute is critical to improving user experience and reducing operational costs in large language model (LLM) services. However, current inference engine schedulers overlook the attention backend's sensitivity to…
Content caching is a widely studied technique aimed to reduce the network load imposed by data transmission during peak time while ensuring users' quality of experience. It has been shown that when there is a common link between caches and…
This paper presents a strategy to allocate services on a Cloud system without overloading nodes and maintaining the system stability with minimum cost. We specify an abstract model of cloud resources utilization, including multiple types of…
We study offline scheduling for large language model (LLM) serving under a fixed KV-cache memory budget, where requests have heterogeneous prompt (prefill) and response (decode) lengths. Prompt tokens determine initial KV usage, and each…
Modern storage systems, often deployed to support multiple tenants in the cloud, must provide performance isolation. Unfortunately, traditional approaches such as fair sharing do not provide performance isolation for storage systems,…
Today's datacenters face important challenges for providing low-latency high-quality interactive services to meet user's expectation. For improving the application throughput, recent research works have embedded application deadline…
Emerging edge computing paradigms enable heterogeneous devices to collaborate on complex computation applications. However, for congestible links and computing units, delay-optimal forwarding and offloading for service chain tasks (e.g.,…