This paper studies introducing viewpoint invariant feature representations in existing action recognition architecture. Despite significant progress in action recognition, efficiently handling geometric variations in large-scale datasets remains challenging. To tackle this problem, we propose a novel Spatial-Temporal Alignment Network (STAN), which explicitly learns geometric invariant representations for action recognition. Notably, the STAN model is light-weighted and generic, which could be plugged into existing action recognition models (e.g., MViTv2) with a low extra computational cost. We test our STAN model on widely-used datasets like UCF101 and HMDB51. The experimental results show that the STAN model can consistently improve the state-of-the-art models in action recognition tasks in trained-from-scratch settings.
@article{arxiv.2308.09897,
title = {Spatial-Temporal Alignment Network for Action Recognition},
author = {Jinhui Ye and Junwei Liang},
journal= {arXiv preprint arXiv:2308.09897},
year = {2023}
}
Comments
Introducing viewpoint invariant feature representations in Action Recognition. arXiv admin note: text overlap with arXiv:2012.02426