基于 DPU 检测 LLM 推理部署中的偏斜、不平衡与病理条件
摘要
Autoregressive inference in large transformer-based language models (LLMs) presents significant challenges for runtime efficiency, particularly during the decode phase where load imbalance across GPU shards can cause throughput degradation and latency spikes. A DPU-assisted framework leveraged by BlueField-3 Data Processing Units can enable real-time detection and mitigation of load imbalance in multi-node tensor-parallel inference. By offloading monitoring tasks to the DPU and analyzing GPU telemetry and inter-node communication patterns, the resulting system can provide actionable feedback to inference controllers and schedulers. The goal of this study is three-fold i) identify the reported skews/imbalances/pathological conditions that arise in muti-GPU execution of a) LLM tensor computing (both during training and inference), b) identify their impact on computational performance, and c) make a critical assessment if those can be tracked for potential mitigation from a DPU network.
引用
@article{arxiv.2509.18114,
title = {A Study of Skews, Imbalances, and Pathological Conditions in LLM Inference Deployment on GPU Clusters detectable from DPU},
author = {Javed I. Khan an Henry Uwabor Moye},
journal= {arXiv preprint arXiv:2509.18114},
year = {2025}
}
备注
12 pages, Technical Report 2025-07-01, Internetworking and Media Communications Research Laboratories, Department of Computer Science, Kent State University