Existing GPU-sharing techniques, including spatial and temporal sharing, aim to improve utilization but face challenges in simultaneously ensuring SLO adherence and maximizing efficiency due to the lack of fine-grained task scheduling on closed-source GPUs. This paper presents Hummingbird, an SLO-oriented GPU scheduling system that overcomes these challenges by enabling microsecond-scale preemption on closed-source GPUs while effectively harvesting idle GPU time slices. Comprehensive evaluations across diverse GPU architectures reveal that Hummingbird improves the SLO attainment of high-priority tasks by 9.7x and 3.5x compared to the state-of-the-art spatial and temporal-sharing approaches. When compared to executing exclusively, the SLO attainment of the high-priority task, collocating with low-priority tasks on Hummingbird, only drops by less than 1%. Meanwhile, the throughput of the low-priority task outperforms the state-of-the-art temporal-sharing approaches by 2.4x. Hummingbird demonstrates significant effectiveness in ensuring the SLO while enhancing GPU utilization.
@article{arxiv.2601.04071,
title = {Hummingbird: SLO-Oriented GPU Preemption at Microsecond-scale},
author = {Tiancheng Hu and Chenxi Wang and Ting Cao and Jin Qin and Lei Chen and Xinyu Xiao and Junhao Hu and Hongliang Tian and Shoumeng Yan and Huimin Cui and Quan Chen and Tao Xie},
journal= {arXiv preprint arXiv:2601.04071},
year = {2026}
}