Shell-Supervised Gaussian Splatting for Urban Real-to-Sim Reconstruction
Abstract
Real-to-sim reconstruction for embodied AI requires geometry that is useful for collision reasoning, navigation, and agent-environment interaction, not only photorealistic novel-view synthesis. However, close-range urban facades are difficult for video-to-3D reconstruction: glass, reflections, repeated windows, and weak texture can produce visually plausible renderings with unstable surface geometry. We introduce shell-supervised Gaussian Splatting, a reconstruction-stage framework that uses an external facade structural shell as lightweight geometric supervision for video-driven Gaussian reconstruction. The method aligns an exterior shell to the video reconstruction frame, renders per-view depth, camera-space normal, and valid-mask maps, and applies these cues through mask-gated losses during Gaussian optimization. This design preserves RGB-driven appearance while regularizing only visible shell-supported facade regions. Experiments on anonymized close-range urban facade scenes show improved facade orientation and visible-surface point-cloud consistency over photo-only, monocular-cue, and surface-oriented Gaussian baselines, while maintaining comparable held-out rendering quality.
Keywords
Cite
@article{arxiv.2606.30014,
title = {Shell-Supervised Gaussian Splatting for Urban Real-to-Sim Reconstruction},
author = {Yuan Yang and Peijun Lu and Fangzhou Lu and Sai Fan and Siqi Yan and Chenyuan Zhang and Haobo Liang and Yichen Wang},
journal= {arXiv preprint arXiv:2606.30014},
year = {2026}
}
Comments
10 pages main paper, 2 pages supplementary material