We in this paper solve the problem of high-quality automatic real-time background cut for 720p portrait videos. We first handle the background ambiguity issue in semantic segmentation by proposing a global background attenuation model. A spatial-temporal refinement network is developed to further refine the segmentation errors in each frame and ensure temporal coherence in the segmentation map. We form an end-to-end network for training and testing. Each module is designed considering efficiency and accuracy. We build a portrait dataset, which includes 8,000 images with high-quality labeled map for training and testing. To further improve the performance, we build a portrait video dataset with 50 sequences to fine-tune video segmentation. Our framework benefits many video processing applications.
@article{arxiv.1704.08812,
title = {Automatic Real-time Background Cut for Portrait Videos},
author = {Xiaoyong Shen and Ruixing Wang and Hengshuang Zhao and Jiaya Jia},
journal= {arXiv preprint arXiv:1704.08812},
year = {2017}
}