English

Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving

Computer Vision and Pattern Recognition 2026-05-28 v5

Abstract

This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving: world representation and world generation. For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries. WorldRec initializes structured queries in 3D space, leveraging them to aggregate cross-view, cross-temporal features, thereby naturally enforcing spatial consistency across frames and yielding compact yet high-fidelity 3D Gaussian scene representations. For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning through three progressive stages (Teacher Forcing, ODE distillation, and DMD), enabling high-quality online causal video generation in as few as 4 denoising steps. Building on both modules, we further introduce the JWM, which deeply integrates WorldRec and WorldGen to achieve synergistic gains in generation stability, cross-frame consistency, and visual fidelity, providing a solid foundation for closed-loop simulation, data synthesis, and end-to-end training in autonomous driving.

Keywords

Cite

@article{arxiv.2605.18137,
  title  = {Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving},
  author = {Lijun Zhou and Hongcheng Luo and Zhenxin Zhu and Cheng Chi and Mingfei Tu and Kaixin Xiong and Lei Gong and Zhanqian Wu and Zehan Zhang and Fangzhen Li and Hao Li and Yingying Shen and Jiale He and Haohui Zhu and Shan Zhao and Kai Wang and Zhiwei Zhan and Yuechuan Pu and Kaiyuan Tan and Ruiling Yang and Xianqi Wang and Tianyi Yan and Jiawei Zhou and Lei Zhang and Jingyang Zhao and Xi Zhou and Chitian Sun and Chenming Wu and Jiong Deng and Hongwei Xie and Ming Lu and Kun Ma and Long Chen and Guang Chen and Hangjun Ye and Bing Wang and Haiyang Sun},
  journal= {arXiv preprint arXiv:2605.18137},
  year   = {2026}
}