Humans perform everyday tasks using a combination of locomotion and manipulation skills. Building a system that can handle both skills is essential to creating virtual humans. We present a physically-simulated human capable of solving box rearrangement tasks, which requires a combination of both skills. We propose a hierarchical control architecture, where each level solves the task at a different level of abstraction, and the result is a physics-based simulated virtual human capable of rearranging boxes in a cluttered environment. The control architecture integrates a planner, diffusion models, and physics-based motion imitation of sparse motion clips using deep reinforcement learning. Boxes can vary in size, weight, shape, and placement height. Code and trained control policies are provided.
@article{arxiv.2306.09532,
title = {Hierarchical Planning and Control for Box Loco-Manipulation},
author = {Zhaoming Xie and Jonathan Tseng and Sebastian Starke and Michiel van de Panne and C. Karen Liu},
journal= {arXiv preprint arXiv:2306.09532},
year = {2023}
}