English

Joint 3D Geometry Reconstruction and Motion Generation for 4D Synthesis from a Single Image

Computer Vision and Pattern Recognition 2025-12-05 v1

Abstract

Generating interactive and dynamic 4D scenes from a single static image remains a core challenge. Most existing generate-then-reconstruct and reconstruct-then-generate methods decouple geometry from motion, causing spatiotemporal inconsistencies and poor generalization. To address these, we extend the reconstruct-then-generate framework to jointly perform Motion generation and geometric Reconstruction for 4D Synthesis (MoRe4D). We first introduce TrajScene-60K, a large-scale dataset of 60,000 video samples with dense point trajectories, addressing the scarcity of high-quality 4D scene data. Based on this, we propose a diffusion-based 4D Scene Trajectory Generator (4D-STraG) to jointly generate geometrically consistent and motion-plausible 4D point trajectories. To leverage single-view priors, we design a depth-guided motion normalization strategy and a motion-aware module for effective geometry and dynamics integration. We then propose a 4D View Synthesis Module (4D-ViSM) to render videos with arbitrary camera trajectories from 4D point track representations. Experiments show that MoRe4D generates high-quality 4D scenes with multi-view consistency and rich dynamic details from a single image. Code: https://github.com/Zhangyr2022/MoRe4D.

Keywords

Cite

@article{arxiv.2512.05044,
  title  = {Joint 3D Geometry Reconstruction and Motion Generation for 4D Synthesis from a Single Image},
  author = {Yanran Zhang and Ziyi Wang and Wenzhao Zheng and Zheng Zhu and Jie Zhou and Jiwen Lu},
  journal= {arXiv preprint arXiv:2512.05044},
  year   = {2025}
}

Comments

18 Pages

R2 v1 2026-07-01T08:09:56.980Z