PhysQuantAgent：面向视觉语言模型的质量估计推理管线
摘要
视觉语言模型(VLM)正在应用于机器人感知与操作，但其推理物理属性（如质量）以完成操作的能力仍受限。 accurately estimating the mass of real-world objects is essential for determining appropriate grasp force and ensuring safe interaction. However, current VLMs lack reliable mass reasoning capabilities, and most existing benchmarks do not explicitly evaluate physical quantity estimation under realistic sensing conditions. In this work, we propose PhysQuantAgent, a framework for real-world object mass estimation using VLMs, together with VisPhysQuant, a new benchmark dataset for evaluation. VisPhysQuant consists of RGB-D videos of real objects captured from multiple viewpoints, annotated with precise mass measurements. To improve estimation accuracy, we introduce three visual prompting methods that enhance the input image with object detection, scale estimation, and cross-sectional image generation to help the model comprehend the size and internal structure of the target object. Experiments show that visual prompting significantly improves mass estimation accuracy on real-world data, suggesting the efficacy of integrating spatial reasoning with VLM knowledge for physical inference.
引用
@article{arxiv.2603.16958,
title = {PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Models},
author = {Hisayuki Yokomizo and Taiki Miyanishi and Yan Gang and Shuhei Kurita and Nakamasa Inoue and Yusuke Iwasawa},
journal= {arXiv preprint arXiv:2603.16958},
year = {2026}
}
备注
Code and dataset will be available at https://github.com/hisasnow/PhysQuantAgent