We tackle the problem of developing humanoid loco-manipulation skills with deep imitation learning. The difficulty of collecting task demonstrations and training policies for humanoids with a high degree of freedom presents substantial challenges. We introduce TRILL, a data-efficient framework for training humanoid loco-manipulation policies from human demonstrations. In this framework, we collect human demonstration data through an intuitive Virtual Reality (VR) interface. We employ the whole-body control formulation to transform task-space commands by human operators into the robot's joint-torque actuation while stabilizing its dynamics. By employing high-level action abstractions tailored for humanoid loco-manipulation, our method can efficiently learn complex sensorimotor skills. We demonstrate the effectiveness of TRILL in simulation and on a real-world robot for performing various loco-manipulation tasks. Videos and additional materials can be found on the project page: https://ut-austin-rpl.github.io/TRILL.
@article{arxiv.2309.01952,
title = {Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperation},
author = {Mingyo Seo and Steve Han and Kyutae Sim and Seung Hyeon Bang and Carlos Gonzalez and Luis Sentis and Yuke Zhu},
journal= {arXiv preprint arXiv:2309.01952},
year = {2023}
}