Deep learning within the context of point clouds has gained much research interest in recent years mostly due to the promising results that have been achieved on a number of challenging benchmarks, such as 3D shape recognition and scene semantic segmentation. In many realistic settings however, snapshots of the environment are often taken from a single view, which only contains a partial set of the scene due to the field of view restriction of commodity cameras. 3D scene semantic understanding on partial point clouds is considered as a challenging task. In this work, we propose a processing approach for 3D point cloud data based on a multiview representation of the existing 360{\deg} point clouds. By fusing the original 360{\deg} point clouds and their corresponding 3D multiview representations as input data, a neural network is able to recognize partial point sets while improving the general performance on complete point sets, resulting in an overall increase of 31.9% and 4.3% in segmentation accuracy for partial and complete scene semantic understanding, respectively. This method can also be applied in a wider 3D recognition context such as 3D part segmentation.
@article{arxiv.1812.01712,
title = {Multiview Based 3D Scene Understanding On Partial Point Sets},
author = {Ye Zhu and Sven Ewan Shepstone and Pablo Martínez-Nuevo and Miklas Strøm Kristoffersen and Fabien Moutarde and Zhuang Fu},
journal= {arXiv preprint arXiv:1812.01712},
year = {2018}
}
Comments
This paper has been submitted to IEEE Transactions on Neural Networks and Learning Systems