A Proposal-Based Solution to Spatio-Temporal Action Detection in Untrimmed Videos
Abstract
Existing approaches for spatio-temporal action detection in videos are limited by the spatial extent and temporal duration of the actions. In this paper, we present a modular system for spatio-temporal action detection in untrimmed security videos. We propose a two stage approach. The first stage generates dense spatio-temporal proposals using hierarchical clustering and temporal jittering techniques on frame-wise object detections. The second stage is a Temporal Refinement I3D (TRI-3D) network that performs action classification and temporal refinement on the generated proposals. The object detection-based proposal generation step helps in detecting actions occurring in a small spatial region of a video frame, while temporal jittering and refinement helps in detecting actions of variable lengths. Experimental results on the spatio-temporal action detection dataset - DIVA - show the effectiveness of our system. For comparison, the performance of our system is also evaluated on the THUMOS14 temporal action detection dataset.
Cite
@article{arxiv.1811.08496,
title = {A Proposal-Based Solution to Spatio-Temporal Action Detection in Untrimmed Videos},
author = {Joshua Gleason and Rajeev Ranjan and Steven Schwarcz and Carlos D. Castillo and Jun-Chen Cheng and Rama Chellappa},
journal= {arXiv preprint arXiv:1811.08496},
year = {2018}
}
Comments
To appear in IEEE Winter Conference on Applications of Computer Vision (WACV) 2019