We leverage unsupervised learning of depth, egomotion, and camera intrinsics to improve the performance of single-image semantic segmentation, by enforcing 3D-geometric and temporal consistency of segmentation masks across video frames. The predicted depth, egomotion, and camera intrinsics are used to provide an additional supervision signal to the segmentation model, significantly enhancing its quality, or, alternatively, reducing the number of labels the segmentation model needs. Our experiments were performed on the ScanNet dataset.
@article{arxiv.2004.05324,
title = {Improving Semantic Segmentation through Spatio-Temporal Consistency Learned from Videos},
author = {Ankita Pasad and Ariel Gordon and Tsung-Yi Lin and Anelia Angelova},
journal= {arXiv preprint arXiv:2004.05324},
year = {2020}
}
Comments
Learning from Unlabeled Videos, CVPR Workshop, 2020