In-Network Collective Operations: Game Changer or Challenge for AI Workloads?

Torsten Hoefler; Mikhail Khalilov; Josiah Clark; Surendra Anubolu; Mohan Kalkunte; Karen Schramm; Eric Spada; Duncan Roweth; Keith Underwood; Adrian Caulfield; Abdul Kabbani; Amirreza Rastegari

doi:10.1109/MC.2025.3616048

In-Network Collective Operations: Game Changer or Challenge for AI Workloads?

Networking and Internet Architecture 2026-01-28 v1 Artificial Intelligence Hardware Architecture Performance Systems and Control Systems and Control

Authors: Torsten Hoefler , Mikhail Khalilov , Josiah Clark , Surendra Anubolu , Mohan Kalkunte , Karen Schramm , Eric Spada , Duncan Roweth , Keith Underwood , Adrian Caulfield , Abdul Kabbani , Amirreza Rastegari

View on arXiv ↗ PDF ↗ DOI ↗

Abstract

This paper summarizes the opportunities of in-network collective operations (INC) for accelerated collective operations in AI workloads. We provide sufficient detail to make this important field accessible to non-experts in AI or networking, fostering a connection between these communities. Consider two types of INC: Edge-INC, where the system is implemented at the node level, and Core-INC, where the system is embedded within network switches. We outline the potential performance benefits as well as six key obstacles in the context of both Edge-INC and Core-INC that may hinder their adoption. Finally, we present a set of predictions for the future development and application of INC.

Cite

@article{arxiv.2601.19132,
  title  = {In-Network Collective Operations: Game Changer or Challenge for AI Workloads?},
  author = {Torsten Hoefler and Mikhail Khalilov and Josiah Clark and Surendra Anubolu and Mohan Kalkunte and Karen Schramm and Eric Spada and Duncan Roweth and Keith Underwood and Adrian Caulfield and Abdul Kabbani and Amirreza Rastegari},
  journal= {arXiv preprint arXiv:2601.19132},
  year   = {2026}
}

In-Network Collective Operations: Game Changer or Challenge for AI Workloads?

Abstract

Cite

Related papers