English

COME: Adding Scene-Centric Forecasting Control to Occupancy World Model

Computer Vision and Pattern Recognition 2025-06-17 v1

Abstract

World models are critical for autonomous driving to simulate environmental dynamics and generate synthetic data. Existing methods struggle to disentangle ego-vehicle motion (perspective shifts) from scene evolvement (agent interactions), leading to suboptimal predictions. Instead, we propose to separate environmental changes from ego-motion by leveraging the scene-centric coordinate systems. In this paper, we introduce COME: a framework that integrates scene-centric forecasting Control into the Occupancy world ModEl. Specifically, COME first generates ego-irrelevant, spatially consistent future features through a scene-centric prediction branch, which are then converted into scene condition using a tailored ControlNet. These condition features are subsequently injected into the occupancy world model, enabling more accurate and controllable future occupancy predictions. Experimental results on the nuScenes-Occ3D dataset show that COME achieves consistent and significant improvements over state-of-the-art (SOTA) methods across diverse configurations, including different input sources (ground-truth, camera-based, fusion-based occupancy) and prediction horizons (3s and 8s). For example, under the same settings, COME achieves 26.3% better mIoU metric than DOME and 23.7% better mIoU metric than UniScene. These results highlight the efficacy of disentangled representation learning in enhancing spatio-temporal prediction fidelity for world models. Code and videos will be available at https://github.com/synsin0/COME.

Keywords

Cite

@article{arxiv.2506.13260,
  title  = {COME: Adding Scene-Centric Forecasting Control to Occupancy World Model},
  author = {Yining Shi and Kun Jiang and Qiang Meng and Ke Wang and Jiabao Wang and Wenchao Sun and Tuopu Wen and Mengmeng Yang and Diange Yang},
  journal= {arXiv preprint arXiv:2506.13260},
  year   = {2025}
}
R2 v1 2026-07-01T03:19:15.242Z