This paper proposes a deep neural architecture, PlaneRCNN, that detects and reconstructs piecewise planar surfaces from a single RGB image. PlaneRCNN employs a variant of Mask R-CNN to detect planes with their plane parameters and segmentation masks. PlaneRCNN then jointly refines all the segmentation masks with a novel loss enforcing the consistency with a nearby view during training. The paper also presents a new benchmark with more fine-grained plane segmentations in the ground-truth, in which, PlaneRCNN outperforms existing state-of-the-art methods with significant margins in the plane detection, segmentation, and reconstruction metrics. PlaneRCNN makes an important step towards robust plane extraction, which would have an immediate impact on a wide range of applications including Robotics, Augmented Reality, and Virtual Reality.
@article{arxiv.1812.04072,
title = {PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image},
author = {Chen Liu and Kihwan Kim and Jinwei Gu and Yasutaka Furukawa and Jan Kautz},
journal= {arXiv preprint arXiv:1812.04072},
year = {2019}
}