Mixture-of-Experts (MoE) models challenge serving infrastructures with dynamic, sparse expert utilization, causing instability on conventional systems designed for dense architectures. We propose EaaS, a novel serving system to enable efficient, scalable, and robust MoE deployment. Our system disaggregates MoE modules into independent, stateless services. This design enables fine-grained resource scaling and provides inherent fault tolerance by decoupling compute units. The architecture is powered by a high-performance, CPU-free peer-to-peer communication library that ensures minimal overhead and high throughput. Experiments confirm EaaS's scalability and efficiency, achieving performance comparable to monolithic systems while providing robust fault tolerance and strong scalability. EaaS incurs less than a 2% throughput reduction under simulated hardware failures that would otherwise halt monolithic architectures. It further saves up to 37.5% of computing resources through dynamic fine-grained adaptation to serving traffic, demonstrating strong resilience for large-scale MoE deployment in production.
@article{arxiv.2509.17863,
title = {Expert-as-a-Service: Towards Efficient, Scalable, and Robust Large-scale MoE Serving},
author = {Ziming Liu and Boyu Tian and Guoteng Wang and Zhen Jiang and Peng Sun and Zhenhua Han and Tian Tang and Xiaohe Hu and Yanmin Jia and Yan Zhang and He Liu and Mingjun Zhang and Yiqi Zhang and Qiaoling Chen and Shenggan Cheng and Mingyu Gao and Yang You and Siyuan Feng},
journal= {arXiv preprint arXiv:2509.17863},
year = {2025}
}