We describe a mobile manipulation hardware and software system capable of autonomously performing complex human-level tasks in real homes, after being taught the task with a single demonstration from a person in virtual reality. This is enabled by a highly capable mobile manipulation robot, whole-body task space hybrid position/force control, teaching of parameterized primitives linked to a robust learned dense visual embeddings representation of the scene, and a task graph of the taught behaviors. We demonstrate the robustness of the approach by presenting results for performing a variety of tasks, under different environmental conditions, in multiple real homes. Our approach achieves 85% overall success rate on three tasks that consist of an average of 45 behaviors each.
@article{arxiv.1910.00127,
title = {A Mobile Manipulation System for One-Shot Teaching of Complex Tasks in Homes},
author = {Max Bajracharya and James Borders and Dan Helmick and Thomas Kollar and Michael Laskey and John Leichty and Jeremy Ma and Umashankar Nagarajan and Akiyoshi Ochiai and Josh Petersen and Krishna Shankar and Kevin Stone and Yutaka Takaoka},
journal= {arXiv preprint arXiv:1910.00127},
year = {2020}
}
Comments
The video is available at: https://youtu.be/HSyAGMGikLk. 7 pages, 5 figures, accepted by IEEE 2020 Robotics International Conference on Robotics and Automation (ICRA)