English

M$^\text{4}$World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming

Computer Vision and Pattern Recognition 2026-07-15 v1 Robotics

Abstract

Driving-world generation has emerged as a core capability for scalable autonomous-driving simulation, yet existing methods remain limited in object-level controllability and long-horizon stability. We present M4^\text{4}World, a Multi-view and Multimodal generative driving world model that synthesizes future surround-view video streams and synchronized LiDAR scans while supporting interactive object Manipulation and stable Minute-long streaming. Fine-grained object manipulation is realized through a flexible conditioning interface that supports explicit control over both the spatial layout and visual appearance of individual objects. Stable minute-long streaming, on the other hand, is achieved through a multi-stage training framework that enables online causal generation in only four denoising steps while maintaining coherent world dynamics throughout extended rollouts. Building on these components, we introduce an efficient few-clip post-training as well as a suite of visual reference-conditioned generation models, preserving general generation ability while allowing rare-case customization for long-tail controllability. To assess controllability beyond realism, we further introduce an automated VLM-based judging pipeline that evaluates scene-level condition adherence, view-wise object controllability, and cross-view object consistency. Comprehensive experiments show that M4^\text{4}World consistently delivers high generation quality, precise controllability, and stable minute-long streaming. Together with downstream long-tail augmentation and scene editing, these results demonstrate the potential of M4^\text{4}World for controllable, scalable driving simulation.

Cite

@article{arxiv.2607.14005,
  title  = {M$^\text{4}$World: A Multi-view Multimodal Driving World Model for Interactive Object Manipulation and Minute-long Streaming},
  author = {Ke Cheng and Hanqiao Ye and Lei Shi and Yahui Liu and Yunhan Shen and Jingtao Dong and Zhenke Wang and Wenxuan Ao and Weixiang Xu and Kaining Huang and Shuhan Shen},
  journal= {arXiv preprint arXiv:2607.14005},
  year   = {2026}
}

Comments

24 pages, 13 figures