ERNIE 5.0 Technical Report
Computation and Language
2026-02-05 v1
Abstract
In this report, we introduce ERNIE 5.0, a natively autoregressive foundation model desinged for unified multimodal understanding and generation across text, image, video, and audio. All modalities are trained from scratch under a unified next-group-of-tokens prediction objective, based on an ultra-sparse mixture-of-experts (MoE) architecture with modality-agnostic expert routing. To address practical challenges in large-scale deployment under diverse resource constraints, ERNIE 5.0 adopts a novel elastic training paradigm. Within a single pre-training run, the model learns a family of sub-models with varying depths, expert capacities, and routing sparsity, enabling flexible trade-offs among performance, model size, and inference latency in memory- or time-constrained scenarios. Moreover, we systematically address the challenges of scaling reinforcement learning to unified foundation models, thereby guaranteeing efficient and stable post-training under ultra-sparse MoE architectures and diverse multimodal settings. Extensive experiments demonstrate that ERNIE 5.0 achieves strong and balanced performance across multiple modalities. To the best of our knowledge, among publicly disclosed models, ERNIE 5.0 represents the first production-scale realization of a trillion-parameter unified autoregressive model that supports both multimodal understanding and generation. To facilitate further research, we present detailed visualizations of modality-agnostic expert routing in the unified model, alongside comprehensive empirical analysis of elastic training, aiming to offer profound insights to the community.
Cite
@article{arxiv.2602.04705,
title = {ERNIE 5.0 Technical Report},
author = {Haifeng Wang and Hua Wu and Tian Wu and Yu Sun and Jing Liu and Dianhai Yu and Yanjun Ma and Jingzhou He and Zhongjun He and Dou Hong and Qiwen Liu and Shuohuan Wang and Junyuan Shang and Zhenyu Zhang and Yuchen Ding and Jinle Zeng and Jiabin Yang and Liang Shen and Ruibiao Chen and Weichong Yin and Siyu Ding and Dai Dai and Shikun Feng and Siqi Bao and Bolei He and Yan Chen and Zhenyu Jiao and Ruiqing Zhang and Zeyu Chen and Qingqing Dang and Kaipeng Deng and Jiajun Jiang and Enlei Gong and Guoxia Wang and Yanlin Sha and Yi Liu and Yehan Zheng and Weijian Xu and Jiaxiang Liu and Zengfeng Zeng and Yingqi Qu and Zhongli Li and Zhengkun Zhang and Xiyang Wang and Zixiang Xu and Xinchao Xu and Zhengjie Huang and Dong Wang and Bingjin Chen and Yue Chang and Xing Yuan and Shiwei Huang and Qiao Zhao and Xinzhe Ding and Shuangshuang Qiao and Baoshan Yang and Bihong Tang and Bin Li and Bingquan Wang and Binhan Tang and Binxiong Zheng and Bo Cui and Bo Ke and Bo Zhang and Bowen Zhang and Boyan Zhang and Boyang Liu and Caiji Zhang and Can Li and Chang Xu and Chao Pang and Chao Zhang and Chaoyi Yuan and Chen Chen and Cheng Cui and Chenlin Yin and Chun Gan and Chunguang Chai and Chuyu Fang and Cuiyun Han and Dan Zhang and Danlei Feng and Danxiang Zhu and Dong Sun and Dongbo Li and Dongdong Li and Dongdong Liu and Dongxue Liu and Fan Ding and Fan Hu and Fan Li and Fan Mo and Feisheng Wu and Fengwei Liu and Gangqiang Hu and Gaofeng Lu and Gaopeng Yong and Gexiao Tian and Guan Wang and Guangchen Ni and Guangshuo Wu and Guanzhong Wang and Guihua Liu and Guishun Li and Haibin Li and Haijian Liang and Haipeng Ming and Haisu Wang and Haiyang Lu and Haiye Lin and Han Zhou and Hangting Lou and Hanwen Du and Hanzhi Zhang and Hao Chen and Hao Du and Hao Liu and Hao Zhou and Haochen Jiang and Haodong Tian and Haoshuang Wang and Haozhe Geng and Heju Yin and Hong Chen and Hongchen Xue and Hongen Liu and Honggeng Zhang and Hongji Xu and Hongwei Chen and Hongyang Zhang and Hongyuan Zhang and Hua Lu and Huan Chen and Huan Wang and Huang He and Hui Liu and Hui Zhong and Huibin Ruan and Jiafeng Lu and Jiage Liang and Jiahao Hu and Jiahao Hu and Jiajie Yang and Jialin Li and Jian Chen and Jian Wu and Jianfeng Yang and Jianguang Jiang and Jianhua Wang and Jianye Chen and Jiaodi Liu and Jiarui Zhou and Jiawei Lv and Jiaxin Zhou and Jiaxuan Liu and Jie Han and Jie Sun and Jiefan Fang and Jihan Liu and Jihua Liu and Jing Hu and Jing Qian and Jing Yan and Jingdong Du and Jingdong Wang and Jingjing Wu and Jingyong Li and Jinheng Wang and Jinjin Li and Jinliang Lu and Jinlin Yu and Jinnan Liu and Jixiang Feng and Jiyi Huang and Jiyuan Zhang and Jun Liang and Jun Xia and Jun Yu and Junda Chen and Junhao Feng and Junhong Xiang and Junliang Li and Kai Liu and Kailun Chen and Kairan Su and Kang Hu and Kangkang Zhou and Ke Chen and Ke Wei and Kui Huang and Kun Wu and Kunbin Chen and Lei Han and Lei Sun and Lei Wen and Linghui Meng and Linhao Yu and Liping Ouyang and Liwen Zhang and Longbin Ji and Longzhi Wang and Meng Sun and Meng Tian and Mengfei Li and Mengqi Zeng and Mengyu Zhang and Ming Hong and Mingcheng Zhou and Mingming Huang and Mingxin Chen and Mingzhu Cai and Naibin Gu and Nemin Qiu and Nian Wang and Peng Qiu and Peng Zhao and Pengyu Zou and Qi Wang and Qi Xin and Qian Wang and Qiang Zhu and Qianhui Luo and Qianwei Yang and Qianyue He and Qifei Wu and Qinrui Li and Qiwen Bao and Quan Zhang and Quanxiang Liu and Qunyi Xie and Rongrui Zhan and Rufeng Dai and Rui Peng and Ruian Liu and Ruihao Xu and Ruijie Wang and Ruixi Zhang and Ruixuan Liu and Runsheng Shi and Ruting Wang and Senbo Kang and Shan Lu and Shaofei Yu and Shaotian Gong and Shenwei Hu and Shifeng Zheng and Shihao Guo and Shilong Fan and Shiqin Liu and Shiwei Gu and Shixi Zhang and Shuai Yao and Shuang Zhang and Shuangqiao Liu and Shuhao Liang and Shuwei He and Shuwen Yang and Sijun He and Siming Dai and Siming Wu and Siyi Long and Songhe Deng and Suhui Dong and Suyin Liang and Teng Hu and Tianchan Xu and Tianliang Lv and Tianmeng Yang and Tianyi Wei and Tiezhu Gao and Ting Sun and Ting Zhang and Tingdan Luo and Wei He and Wei Luan and Wei Yin and Wei Zhang and Wei Zhou and Weibao Gong and Weibin Li and Weicheng Huang and Weichong Dang and Weiguo Zhu and Weilong Zhang and Weiqi Tan and Wen Huang and Wenbin Chang and Wenjing Du and Wenlong Miao and Wenpei Luo and Wenquan Wu and Xi Shi and Xi Zhao and Xiang Gao and Xiangguo Zhang and Xiangrui Yu and Xiangsen Wang and Xiangzhe Wang and Xianlong Luo and Xianying Ma and Xiao Tan and Xiaocong Lin and Xiaofei Wang and Xiaofeng Peng and Xiaofeng Wu and Xiaojian Xu and Xiaolan Yuan and Xiaopeng Cui and Xiaotian Han and Xiaoxiong Liu and Xiaoxu Fei and Xiaoxuan Wu and Xiaoyu Wang and Xiaoyu Zhang and Xin Sun and Xin Wang and Xinhui Huang and Xinming Zhu and Xintong Yu and Xinyi Xu and Xinyu Wang and Xiuxian Li and XuanShi Zhu and Xue Xu and Xueying Lv and Xuhong Li and Xulong Wei and Xuyi Chen and Yabing Shi and Yafeng Wang and Yamei Li and Yan Liu and Yanfu Cheng and Yang Gao and Yang Liang and Yang Wang and Yang Wang and Yang Yang and Yanlong Liu and Yannian Fu and Yanpeng Wang and Yanzheng Lin and Yao Chen and Yaozong Shen and Yaqian Han and Yehua Yang and Yekun Chai and Yesong Wang and Yi Song and Yichen Zhang and Yifei Wang and Yifeng Guo and Yifeng Kou and Yilong Chen and Yilong Guo and Yiming Wang and Ying Chen and Ying Wang and Yingsheng Wu and Yingzhan Lin and Yinqi Yang and Yiran Xing and Yishu Lei and Yixiang Tu and Yiyan Chen and Yong Zhang and Yonghua Li and Yongqiang Ma and Yongxing Dai and Yongyue Zhang and Yu Ran and Yu Sun and Yu-Wen Michael Zhang and Yuang Liu and Yuanle Liu and Yuanyuan Zhou and Yubo Zhang and Yuchen Han and Yucheng Wang and Yude Gao and Yuedong Luo and Yuehu Dong and Yufeng Hu and Yuhui Cao and Yuhui Yun and Yukun Chen and Yukun Gao and Yukun Li and Yumeng Zhang and Yun Fan and Yun Ma and Yunfei Zhang and Yunshen Xie and Yuping Xu and Yuqin Zhang and Yuqing Liu and Yurui Li and Yuwen Wang and Yuxiang Lu and Zefeng Cai and Zelin Zhao and Zelun Zhang and Zenan Lin and Zezhao Dong and Zhaowu Pan and Zhaoyu Liu and Zhe Dong and Zhe Zhang and Zhen Zhang and Zhengfan Wu and Zhengrui Wei and Zhengsheng Ning and Zhenxing Li and Zhenyu Li and Zhenyu Qian and Zhenyun Li and Zhi Li and Zhichao Chen and Zhicheng Dong and Zhida Feng and Zhifan Feng and Zhihao Deng and Zhijin Yu and Zhiyang Chen and Zhonghui Zheng and Zhuangzhuang Guo and Zhujun Zhang and Zhuo Sun and Zichang Liu and Zihan Lin and Zihao Huang and Zihe Zhu and Ziheng Zhao and Ziping Chen and Zixuan Zhu and Ziyang Xu and Ziyi Liang and Ziyuan Gao},
journal= {arXiv preprint arXiv:2602.04705},
year = {2026}
}