We systematically compare and analyze a set of key components in unsupervised optical flow to identify which photometric loss, occlusion handling, and smoothness regularization is most effective. Alongside this investigation we construct a number of novel improvements to unsupervised flow models, such as cost volume normalization, stopping the gradient at the occlusion mask, encouraging smoothness before upsampling the flow field, and continual self-supervision with image resizing. By combining the results of our investigation with our improved model components, we are able to present a new unsupervised flow technique that significantly outperforms the previous unsupervised state-of-the-art and performs on par with supervised FlowNet2 on the KITTI 2015 dataset, while also being significantly simpler than related approaches.
@article{arxiv.2006.04902,
title = {What Matters in Unsupervised Optical Flow},
author = {Rico Jonschkowski and Austin Stone and Jonathan T. Barron and Ariel Gordon and Kurt Konolige and Anelia Angelova},
journal= {arXiv preprint arXiv:2006.04902},
year = {2020}
}
Comments
Accepted at ECCV 2020 (Oral). Source code is available at https://github.com/google-research/google-research/tree/master/uflow