The performance of prediction-based assistance for robot teleoperation degrades in unseen or goal-rich environments due to incorrect or quickly-changing intent inferences. Poor predictions can confuse operators or cause them to change their control input to implicitly signal their goal. We present a new assistance interface for robotic manipulation where an operator can explicitly communicate a manipulation goal by pointing the end-effector. The pointing target specifies a region for local pose generation and optimization, providing interactive control over grasp and placement pose candidates. We compare the explicit pointing interface to an implicit inference-based assistance scheme in a within-subjects user study (N=20) where participants teleoperate a simulated robot to complete a multi-step singulation and stacking task in cluttered environments. We find that operators prefer the explicit interface, experience fewer pick failures and report lower cognitive workload. Our code is available at: https://github.com/NVlabs/fast-explicit-teleop
@article{arxiv.2402.02612,
title = {Fast Explicit-Input Assistance for Teleoperation in Clutter},
author = {Nick Walker and Xuning Yang and Animesh Garg and Maya Cakmak and Dieter Fox and Claudia Pérez-D'Arpino},
journal= {arXiv preprint arXiv:2402.02612},
year = {2024}
}