Mixture-of-Experts (MoE) has been adopted by many leading large models to reduce computational requirements. However, frequent inter-GPU communication in MoE expert parallelism (EP) becomes a performance challenge. We observe substantial redundant inter-GPU data transfers in MoE that can be potentially addressed by in-switch computing. Unfortunately, the existing solution, NVLink SHARP (NVLS), can only support static collectives with regular patterns, incapable of dynamic communication with irregular patterns in MoE. To bridge the functionality gap, we propose DySHARP, an integral dynamic in-switch computing solution to accelerate MoE, encompassing both communication primitives and communication-aware scheduling: 1) Dynamic multimem addressing co-designs ISA, architecture, and runtime, as a dynamic extension to NVLS, reducing redundant traffic. However, the resulting traffic reduction is inherently asymmetric between two directions, preventing it from directly translating into speedup. 2) Token-centric kernel fusion deeply fuses the dispatch-computation-combine pipeline, resolving this asymmetry to translate traffic reduction into actual speedup. Compared with the state-of-the-art solution, DySHARP achieves up to 1.79× speedup.
@article{arxiv.2605.05607,
title = {Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs},
author = {Qijun Zhang and Chen Zhang and Zhuoshan Zhou and Haibo Wang and Zhe Zhou and Zhipeng Tu and Guangyu Sun and Zhiyao Xie and Yijia Diao and Zhigang Ji and Jingwen Leng and Guanghui He and Minyi Guo},
journal= {arXiv preprint arXiv:2605.05607},
year = {2026}
}