This paper describes our solution for the video recognition task of the Google Cloud and YouTube-8M Video Understanding Challenge that ranked the 3rd place. Because the challenge provides pre-extracted visual and audio features instead of the raw videos, we mainly investigate various temporal modeling approaches to aggregate the frame-level features for multi-label video recognition. Our system contains three major components: two-stream sequence model, fast-forward sequence model and temporal residual neural networks. Experiment results on the challenging Youtube-8M dataset demonstrate that our proposed temporal modeling approaches can significantly improve existing temporal modeling approaches in the large-scale video recognition tasks. To be noted, our fast-forward LSTM with a depth of 7 layers achieves 82.75% in term of GAP@20 on the Kaggle Public test set.
@article{arxiv.1707.04555,
title = {Temporal Modeling Approaches for Large-scale Youtube-8M Video Understanding},
author = {Fu Li and Chuang Gan and Xiao Liu and Yunlong Bian and Xiang Long and Yandong Li and Zhichao Li and Jie Zhou and Shilei Wen},
journal= {arXiv preprint arXiv:1707.04555},
year = {2017}
}
Comments
To appear on CVPR 2017 YouTube-8M Workshop(Rank 3rd out of 650 teams)