Two Stream Self-Supervised Learning for Action Recognition
Abstract
We present a self-supervised approach using spatio-temporal signals between video frames for action recognition. A two-stream architecture is leveraged to tangle spatial and temporal representation learning. Our task is formulated as both a sequence verification and spatio-temporal alignment tasks. The former task requires motion temporal structure understanding while the latter couples the learned motion with the spatial representation. The self-supervised pre-trained weights effectiveness is validated on the action recognition task. Quantitative evaluation shows the self-supervised approach competence on three datasets: HMDB51, UCF101, and Honda driving dataset (HDD). Further investigations to boost performance and generalize validity are still required.
Cite
@article{arxiv.1806.07383,
title = {Two Stream Self-Supervised Learning for Action Recognition},
author = {Ahmed Taha and Moustafa Meshry and Xitong Yang and Yi-Ting Chen and Larry Davis},
journal= {arXiv preprint arXiv:1806.07383},
year = {2018}
}