English

Perception-Aware Multimodal Spatial Reasoning from Monocular Images

Computer Vision and Pattern Recognition 2026-03-10 v1

Abstract

Spatial reasoning from monocular images is essential for autonomous driving, yet current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception, particularly under large scale variation and ambiguous object appearance. We propose a simple yet effective perception-aware multimodal reasoning framework that equips VLMs with explicit object-centric grounding ability. Instead of relying on textual bounding-box outputs, each referred object is represented using all Visual Reference Tokens (VRTs) within its spatial extent, enabling visual evidence and textual reasoning to be processed jointly in a unified token space. To further strengthen cross-modal interaction, we construct a Multimodal Chain-of-Thought (MM-CoT) dataset that injects aligned visual and textual reasoning signals. A deterministic ordering strategy is introduced to make supervision over inherently unordered VRT sets fully compatible with the VLM's autoregressive next-token prediction. With only standard supervised fine-tuning, our method achieves substantial improvements on the SURDS benchmark, outperforming previous approaches - including those using RL-based post-training - by a large margin across both single-object and multi-object tasks. These results demonstrate that accurate perception and multimodal reasoning are mutually reinforcing, and together form the key to robust spatial understanding in challenging monocular driving scenarios.

Keywords

Cite

@article{arxiv.2603.06985,
  title  = {Perception-Aware Multimodal Spatial Reasoning from Monocular Images},
  author = {Yanchun Cheng and Rundong Wang and Xulei Yang and Alok Prakash and Daniela Rus and Marcelo H Ang and ShiJie Li},
  journal= {arXiv preprint arXiv:2603.06985},
  year   = {2026}
}
R2 v1 2026-07-01T11:08:10.265Z