Towards Fair and Firm Real-Time Scheduling in DNN Multi-Tenant Multi-Accelerator Systems via Reinforcement Learning

Enrico Russo; Francesco Giulio Blanco; Maurizio Palesi; Giuseppe Ascia; Davide Patti; Vincenzo Catania

Towards Fair and Firm Real-Time Scheduling in DNN Multi-Tenant Multi-Accelerator Systems via Reinforcement Learning

Hardware Architecture 2024-03-05 v1 Distributed, Parallel, and Cluster Computing Machine Learning

Authors: Enrico Russo , Francesco Giulio Blanco , Maurizio Palesi , Giuseppe Ascia , Davide Patti , Vincenzo Catania

Abstract

This paper addresses the critical challenge of managing Quality of Service (QoS) in cloud services, focusing on the nuances of individual tenant expectations and varying Service Level Indicators (SLIs). It introduces a novel approach utilizing Deep Reinforcement Learning for tenant-specific QoS management in multi-tenant, multi-accelerator cloud environments. The chosen SLI, deadline hit rate, allows clients to tailor QoS for each service request. A novel online scheduling algorithm for Deep Neural Networks in multi-accelerator systems is proposed, with a focus on guaranteeing tenant-wise, model-specific QoS levels while considering real-time constraints.

Keywords

deep reinforcement learning QoS management multi-tenant multi-accelerator real-time scheduling deep neural networks deadline hit rate

Cite

@article{arxiv.2403.00766,
  title  = {Towards Fair and Firm Real-Time Scheduling in DNN Multi-Tenant Multi-Accelerator Systems via Reinforcement Learning},
  author = {Enrico Russo and Francesco Giulio Blanco and Maurizio Palesi and Giuseppe Ascia and Davide Patti and Vincenzo Catania},
  journal= {arXiv preprint arXiv:2403.00766},
  year   = {2024}
}

Towards Fair and Firm Real-Time Scheduling in DNN Multi-Tenant Multi-Accelerator Systems via Reinforcement Learning

Abstract

Keywords

Cite

Related papers