English

Evaluating Pointing Gestures for Target Selection in Human-Robot Collaboration

Robotics 2025-06-30 v1 Computer Vision and Pattern Recognition

Abstract

Pointing gestures are a common interaction method used in Human-Robot Collaboration for various tasks, ranging from selecting targets to guiding industrial processes. This study introduces a method for localizing pointed targets within a planar workspace. The approach employs pose estimation, and a simple geometric model based on shoulder-wrist extension to extract gesturing data from an RGB-D stream. The study proposes a rigorous methodology and comprehensive analysis for evaluating pointing gestures and target selection in typical robotic tasks. In addition to evaluating tool accuracy, the tool is integrated into a proof-of-concept robotic system, which includes object detection, speech transcription, and speech synthesis to demonstrate the integration of multiple modalities in a collaborative application. Finally, a discussion over tool limitations and performance is provided to understand its role in multimodal robotic systems. All developments are available at: https://github.com/NMKsas/gesture_pointer.git.

Keywords

Cite

@article{arxiv.2506.22116,
  title  = {Evaluating Pointing Gestures for Target Selection in Human-Robot Collaboration},
  author = {Noora Sassali and Roel Pieters},
  journal= {arXiv preprint arXiv:2506.22116},
  year   = {2025}
}

Comments

Accepted by the 2025 34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). Preprint

R2 v1 2026-07-01T03:36:14.132Z