In this work we present a self-supervised learning framework to simultaneously train two Convolutional Neural Networks (CNNs) to predict depth and surface normals from a single image. In contrast to most existing frameworks which represent outdoor scenes as fronto-parallel planes at piece-wise smooth depth, we propose to predict depth with surface orientation while assuming that natural scenes have piece-wise smooth normals. We show that a simple depth-normal consistency as a soft-constraint on the predictions is sufficient and effective for training both these networks simultaneously. The trained normal network provides state-of-the-art predictions while the depth network, relying on much realistic smooth normal assumption, outperforms the traditional self-supervised depth prediction network by a large margin on the KITTI benchmark. Demo video: https://youtu.be/ZD-ZRsw7hdM
@article{arxiv.1903.00112,
title = {Self-supervised Learning for Single View Depth and Surface Normal Estimation},
author = {Huangying Zhan and Chamara Saroj Weerasekera and Ravi Garg and Ian Reid},
journal= {arXiv preprint arXiv:1903.00112},
year = {2019}
}