Photorealistic 4D reconstruction of street scenes is essential for developing real-world simulators in autonomous driving. However, most existing methods perform this task offline and rely on time-consuming iterative processes, limiting their practical applications. To this end, we introduce the Large 4D Gaussian Reconstruction Model (DrivingRecon), a generalizable driving scene reconstruction model, which directly predicts 4D Gaussian from surround view videos. To better integrate the surround-view images, the Prune and Dilate Block (PD-Block) is proposed to eliminate overlapping Gaussian points between adjacent views and remove redundant background points. To enhance cross-temporal information, dynamic and static decoupling is tailored to better learn geometry and motion features. Experimental results demonstrate that DrivingRecon significantly improves scene reconstruction quality and novel view synthesis compared to existing methods. Furthermore, we explore applications of DrivingRecon in model pre-training, vehicle adaptation, and scene editing. Our code is available at https://github.com/EnVision-Research/DriveRecon.
@article{arxiv.2412.09043,
title = {DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Driving},
author = {Hao Lu and Tianshuo Xu and Wenzhao Zheng and Yunpeng Zhang and Wei Zhan and Dalong Du and Masayoshi Tomizuka and Kurt Keutzer and Yingcong Chen},
journal= {arXiv preprint arXiv:2412.09043},
year = {2024}
}