Despite online learning (OL) techniques have boosted the performance of semi-supervised video object segmentation (VOS) methods, the huge time costs of OL greatly restrict their practicality. Matching based and propagation based methods run at a faster speed by avoiding OL techniques. However, they are limited by sub-optimal accuracy, due to mismatching and drifting problems. In this paper, we develop a real-time yet very accurate Ranking Attention Network (RANet) for VOS. Specifically, to integrate the insights of matching based and propagation based methods, we employ an encoder-decoder framework to learn pixel-level similarity and segmentation in an end-to-end manner. To better utilize the similarity maps, we propose a novel ranking attention module, which automatically ranks and selects these maps for fine-grained VOS performance. Experiments on DAVIS-16 and DAVIS-17 datasets show that our RANet achieves the best speed-accuracy trade-off, e.g., with 33 milliseconds per frame and J&F=85.5% on DAVIS-16. With OL, our RANet reaches J&F=87.1% on DAVIS-16, exceeding state-of-the-art VOS methods. The code can be found at https://github.com/Storife/RANet.
@article{arxiv.1908.06647,
title = {RANet: Ranking Attention Network for Fast Video Object Segmentation},
author = {Ziqin Wang and Jun Xu and Li Liu and Fan Zhu and Ling Shao},
journal= {arXiv preprint arXiv:1908.06647},
year = {2020}
}
Comments
Accepted by ICCV 2019. 10 pages, 7 figures, 6 tables. The supplementary file can be found at https://csjunxu.github.io/paper/2019ICCV/RANet_supp.pdf ; Code is available at https://github.com/Storife/RANet