An Extensible Software Transport Layer for GPU Networking

Yang Zhou; Zhongjie Chen; Ziming Mao; ChonLam Lao; Shuo Yang; Pravein Govindan Kannan; Jiaqi Gao; Yilong Zhao; Yongji Wu; Kaichao You; Fengyuan Ren; Zhiying Xu; Costin Raiciu; Ion Stoica

An Extensible Software Transport Layer for GPU Networking

Networking and Internet Architecture 2025-08-06 v2

Authors: Yang Zhou , Zhongjie Chen , Ziming Mao , ChonLam Lao , Shuo Yang , Pravein Govindan Kannan , Jiaqi Gao , Yilong Zhao , Yongji Wu , Kaichao You , Fengyuan Ren , Zhiying Xu , Costin Raiciu , Ion Stoica

View on arXiv ↗ PDF ↗

Abstract

Fast-evolving machine learning (ML) workloads have increasing requirements for networking. However, host network transport on RDMA NICs is hard to evolve, causing problems for ML workloads. For example, single-path RDMA traffic is prone to flow collisions that severely degrade collective communication performance. We present UCCL, an extensible software transport layer to evolve GPU networking. UCCL decouples the data path and control path of existing RDMA NICs and efficiently runs the control-path transport on host CPUs. This software extensibility brings in transport innovations that cannot be achieved in hardware for ML workloads, e.g., a multipath transport to resolve flow collisions. ML collectives atop UCCL achieve up to 4.5x higher performance compared to existing RDMA NICs.

Keywords

large language model inference gpu computing network protocol

Cite

@article{arxiv.2504.17307,
  title  = {An Extensible Software Transport Layer for GPU Networking},
  author = {Yang Zhou and Zhongjie Chen and Ziming Mao and ChonLam Lao and Shuo Yang and Pravein Govindan Kannan and Jiaqi Gao and Yilong Zhao and Yongji Wu and Kaichao You and Fengyuan Ren and Zhiying Xu and Costin Raiciu and Ion Stoica},
  journal= {arXiv preprint arXiv:2504.17307},
  year   = {2025}
}

An Extensible Software Transport Layer for GPU Networking

Abstract

Keywords

Cite

Related papers