Robot warehouse automation has attracted significant interest in recent years, perhaps most visibly in the Amazon Picking Challenge (APC). A fully autonomous warehouse pick-and-place system requires robust vision that reliably recognizes and locates objects amid cluttered environments, self-occlusions, sensor noise, and a large variety of objects. In this paper we present an approach that leverages multi-view RGB-D data and self-supervised, data-driven learning to overcome those difficulties. The approach was part of the MIT-Princeton Team system that took 3rd- and 4th- place in the stowing and picking tasks, respectively at APC 2016. In the proposed approach, we segment and label multiple views of a scene with a fully convolutional neural network, and then fit pre-scanned 3D object models to the resulting segmentation to get the 6D object pose. Training a deep neural network for segmentation typically requires a large amount of training data. We propose a self-supervised method to generate a large labeled dataset without tedious manual segmentation. We demonstrate that our system can reliably estimate the 6D pose of objects under a variety of scenarios. All code, data, and benchmarks are available at http://apc.cs.princeton.edu/
@article{arxiv.1609.09475,
title = {Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge},
author = {Andy Zeng and Kuan-Ting Yu and Shuran Song and Daniel Suo and Ed Walker and Alberto Rodriguez and Jianxiong Xiao},
journal= {arXiv preprint arXiv:1609.09475},
year = {2017}
}
Comments
To appear at the International Conference on Robotics and Automation (ICRA) 2017. Project webpage: http://apc.cs.princeton.edu/