HomeComputer VisionarXiv:2605.30307

Grounded 3D-Aware Spatial Vision-Language Modeling

Computer Vision2026-05v1license

Abstract

We present GR3D, a spatial vision language model equipped with three complementary grounding capabilities--explicit 2D grounding, implicit 2D grounding, and monocular 3D grounding--within a single framework. GR3D introduces an implicit grounding mechanism that identifies entity mentions during generation and inserts the corresponding region tokens into the text stream, allowing the model to reference visual evidence on the fly when producing spatial chain-of-thought responses. In parallel, a region-prompted monocular 3D grounding design predicts 3D bounding boxes in the camera view from grounded region queries, supported by intrinsic-aware normalization and dense geometric supervision. Together, these grounding capabilities enable GR3D to decompose complex spatial understanding problems into grounded 2D perception followed by 3D inference. GR3D achieves consistent improvements across grounded and non-grounded spatial benchmarks, demonstrating grounding as an effective inductive bias for strengthening spatial understanding in VLMs. These grounding capabilities collectively enhance general spatial understanding beyond the grounding task itself.

Comments: CVPR 2026 https://www.anjiecheng.me/gr3d

Cite

@article{arxiv.2605.30307,
  title  = {Grounded 3D-Aware Spatial Vision-Language Modeling},
  author = {An-Chieh Cheng and Yang Fu and Yatai Ji and Ligeng Zhu and Guanqi Zhan and Zhuoyang Zhang and Zhaojing Yang and Song Han and Yao Lu and Pavlo Molchanov and Vidya Nariyambut Murali and Jan Kautz and Xiaolong Wang and Hongxu Yin and Sifei Liu},
  journal= {arXiv preprint arXiv:2605.30307},
  year   = {2026}
}