Related papers: Admission Control with Response Time Objectives fo…
Prompt-based offline methods are commonly used to optimize large language model (LLM) responses, but evaluating these responses is computationally intensive and often fails to accommodate diverse response styles. This study introduces a…
Autoscaling is a technology that automatically scales resources for applications without human intervention to ensure runtime Quality of Service (QoS) while reducing costs. However, user-facing cloud applications serve dynamic workloads…
In order to utilize network resources and provide a more reliable delivery of information, multipath routing algorithms are used with a higher priority than single path routing. But providing the quality of service (QoS) requirements while…
Mobility-on-demand (MoD) systems, especially ride-hailing systems, have seen tremendous growth in recent years. These systems provide user-centric mobility services, whose users expect a high level of convenience. Waiting for a response…
The front end bottleneck in datacenter workloads has come under increased scrutiny, with the growing code footprint, involvement of numerous libraries and OS services, and the unpredictability in the instruction stream. Our examination of…
Fog computing is a promising computing paradigm in which IoT data can be processed near the edge to support time-sensitive applications. However, the availability of the resources in the computation device is not stable since they may not…
Large Language Models (LLMs) are increasingly deployed in both latency-sensitive online services and cost-sensitive offline workloads. Co-locating these workloads on shared serving instances can improve resource utilization, but directly…
Nowadays, data-centers are largely under-utilized because resource allocation is based on reservation mechanisms which ignore actual resource utilization. Indeed, it is common to reserve resources for peak demand, which may occur only for a…
Airport public transport systems are plagued by passenger queue congestion, imposing a substandard travel experience and unexpected delays. To address this issue, this paper proposes a bi-level programming for optimizing queueing network in…
In this study, we consider multi-class multi-server asymmetric queueing systems consisting of $N$ queues on one side and $K$ servers on the other side, where jobs randomly arrive in queues at each time. The service rate of each job-server…
We consider a discrete-time parallel service system consisting of $n$ heterogeneous single server queues with infinite capacity. Jobs arrive to the system as an i.i.d. process with rate proportional to $n$, and must be immediately…
Increasing data volumes in scientific experiments necessitate the use of high-performance computing (HPC) resources for data analysis. In many scientific fields, the data generated from scientific instruments and supercomputer simulations…
Modern data centers suffer from immense power consumption. As a result, data center operators have heavily invested in capacity scaling solutions, which dynamically deactivate servers if the demand is low and activate them again when the…
Selecting appropriate values for the configurable parameters of Database Management Systems (DBMS) to improve performance is a significant challenge. Recent machine learning (ML)-based tuning systems have shown strong potential, but their…
Existing Large Language Model (LLM) serving systems prioritize maximum throughput. They often neglect Service Level Objectives (SLOs) such as Time to First Token (TTFT) and Time Per Output Token (TPOT), which leads to suboptimal SLO…
The configuration balancing problem with stochastic requests generalizes many well-studied resource allocation problems such as load balancing and virtual circuit routing. In it, we have $m$ resources and $n$ requests. Each request has…
In an overloaded FaaS cluster, individual worker nodes strain under lengthening queues of requests. Although the cluster might be eventually horizontally-scaled, adding a new node takes dozens of seconds. As serving applications are tuned…
Load Balancing plays a vital role in modern data centers to distribute traffic among instances of network functions or services. State-of-the-art load balancers such as Silkroad dispatch traffic obliviously without considering the real-time…
The rapid proliferation of shared edge computing platforms has enabled application service providers to deploy a wide variety of services with stringent latency and high bandwidth requirements. A key advantage of these platforms is that…
Fog computing is a promising computing paradigm for time-sensitive Internet of Things (IoT) applications. It helps to process data close to the users, in order to deliver faster processing outcomes than the Cloud; it also helps to reduce…