English

Pointing-Based Object Recognition

Computer Vision and Pattern Recognition 2026-03-17 v1

Abstract

This paper presents a comprehensive pipeline for recognizing objects targeted by human pointing gestures using RGB images. As human-robot interaction moves toward more intuitive interfaces, the ability to identify targets of non-verbal communication becomes crucial. Our proposed system integrates several existing state-of-the-art methods, including object detection, body pose estimation, monocular depth estimation, and vision-language models. We evaluate the impact of 3D spatial information reconstructed from a single image and the utility of image captioning models in correcting classification errors. Experimental results on a custom dataset show that incorporating depth information significantly improves target identification, especially in complex scenes with overlapping objects. The modularity of the approach allows for deployment in environments where specialized depth sensors are unavailable.

Keywords

Cite

@article{arxiv.2603.15403,
  title  = {Pointing-Based Object Recognition},
  author = {Lukáš Hajdúch and Viktor Kocur},
  journal= {arXiv preprint arXiv:2603.15403},
  year   = {2026}
}

Comments

Submitted to InnovAIte conference

R2 v1 2026-07-01T11:22:28.567Z