Gaze-based dual resolution deep imitation learning for high-precision dexterous robot manipulation
Abstract
A high-precision manipulation task, such as needle threading, is challenging. Physiological studies have proposed connecting low-resolution peripheral vision and fast movement to transport the hand into the vicinity of an object, and using high-resolution foveated vision to achieve the accurate homing of the hand to the object. The results of this study demonstrate that a deep imitation learning based method, inspired by the gaze-based dual resolution visuomotor control system in humans, can solve the needle threading task. First, we recorded the gaze movements of a human operator who was teleoperating a robot. Then, we used only a high-resolution image around the gaze to precisely control the thread position when it was close to the target. We used a low-resolution peripheral image to reach the vicinity of the target. The experimental results obtained in this study demonstrate that the proposed method enables precise manipulation tasks using a general-purpose robot manipulator and improves computational efficiency. Data from this and related works are available at: https://sites.google.com/view/multi-task-fine.
Cite
@article{arxiv.2102.01295,
title = {Gaze-based dual resolution deep imitation learning for high-precision dexterous robot manipulation},
author = {Heecheol Kim and Yoshiyuki Ohmura and Yasuo Kuniyoshi},
journal= {arXiv preprint arXiv:2102.01295},
year = {2025}
}
Comments
8 pages. The supplementary video can be found at: https://www.youtube.com/watch?v=ytpChcFqD5g Published in IEEE Robotics and Automation Letters. Replaced to add video url in the manuscript