Video scene parsing in the wild with diverse scenarios is a challenging and great significance task, especially with the rapid development of automatic driving technique. The dataset Video Scene Parsing in the Wild(VSPW) contains well-trimmed long-temporal, dense annotation and high resolution clips. Based on VSPW, we design a Temporal Bilateral Network with Vision Transformer. We first design a spatial path with convolutions to generate low level features which can preserve the spatial information. Meanwhile, a context path with vision transformer is employed to obtain sufficient context information. Furthermore, a temporal context module is designed to harness the inter-frames contextual information. Finally, the proposed method can achieve the mean intersection over union(mIoU) of 49.85\% for the VSPW2021 Challenge test dataset.
@article{arxiv.2112.01033,
title = {TBN-ViT: Temporal Bilateral Network with Vision Transformer for Video Scene Parsing},
author = {Bo Yan and Leilei Cao and Hongbin Wang},
journal= {arXiv preprint arXiv:2112.01033},
year = {2021}
}
Comments
The sixth place solution for ICCV2021 VSPW Challenge