Pretraining 3D encoders by aligning with Contrastive Language Image Pretraining (CLIP) has emerged as a promising direction to learn generalizable representations for 3D scene understanding. In this paper, we propose UniScene3D, a transformer-based encoder that learns unified scene representations from multi-view colored pointmaps, jointly modeling image appearance and geometry. For robust colored pointmap representation learning, we introduce novel cross-view geometric alignment and grounded view alignment to enforce cross-view geometry and semantic consistency. Extensive low-shot and task-specific fine-tuning evaluations on viewpoint grounding, scene retrieval, scene type classification, and 3D VQA demonstrate our state-of-the-art performance. These results highlight the effectiveness of our approach for unified 3D scene understanding. https://yebulabula.github.io/UniScene3D/
@article{arxiv.2604.02546,
title = {Contrastive Language-Colored Pointmap Pretraining for Unified 3D Scene Understanding},
author = {Ye Mao and Weixun Luo and Ranran Huang and Junpeng Jing and Krystian Mikolajczyk},
journal= {arXiv preprint arXiv:2604.02546},
year = {2026}
}