We introduce Yuan3.0 Ultra, an open-source Mixture-of-Experts (MoE) large language model featuring 68.8B activated parameters and 1010B total parameters, specially designed to enhance performance on enterprise scenarios tasks while maintaining competitive capabilities on general purpose tasks. We propose Layer-Adaptive Expert Pruning (LAEP) algorithm designed for the pre-training stage of MoE LLMs. In contrast to previous expert pruning approaches that operate primarily in the post-training phase, the proposed algorithm enhances training efficiency by selectively pruning underutilized experts and reorganizing experts across computing devices according to token distribution statistics. Comprehensive experiments demonstrate that LAEP effectively reduces model size and substantially improves pre-training efficiency. When pre-training Yuan3.0 Ultra from scratch original with 1515B parameters, this algorithm delivers a 49\% boost in pre-training efficiency and a 33.3\% reduction in total parameters, while preserving the model's outstanding multi-domain performance. On enterprise scenario benchmarks including Docmatix, ChatRAG, SummEval and MMTab, Yuan3.0 Ultra achieves leading accuracy. The model and codes are publicly available at https://github.com/Yuan-lab-LLM/Yuan3.0-Ultra.
@article{arxiv.2601.14327,
title = {Yuan3.0 Ultra: A Trillion-Parameter Enterprise-Oriented MoE LLM},
author = {YuanLab. ai and : and Shawn Wu and Jiangang Luo and Darcy Chen and Sean Wang and Louie Li and Allen Wang and Xudong Zhao and Tong Yu and Bach Li and Joseph Shen and Gawain Ma and Jasper Jia and Marcus Mao and Claire Wang and Hunter He and Carol Wang and Zera Zhang and Jason Wang and Chonly Shen and Leo Zhang and Logan Chen and Qasim Meng and James Gong and Daniel Zhao and Penn Zheng and Owen Zhu},
journal= {arXiv preprint arXiv:2601.14327},
year = {2026}
}