In this report, we introduce our adaptation of image-text models for long-term action anticipation. Our Video + CLIP framework makes use of a large-scale pre-trained paired image-text model: CLIP and a video encoder Slowfast network. The CLIP embedding provides fine-grained understanding of objects relevant for an action whereas the slowfast network is responsible for modeling temporal information within a video clip of few frames. We show that the features obtained from both encoders are complementary to each other, thus outperforming the baseline on Ego4D for the task of long-term action anticipation. Our code is available at github.com/srijandas07/clip_baseline_LTA_Ego4d.
@article{arxiv.2207.00579,
title = {Video + CLIP Baseline for Ego4D Long-term Action Anticipation},
author = {Srijan Das and Michael S. Ryoo},
journal= {arXiv preprint arXiv:2207.00579},
year = {2022}
}
Comments
Secured second position in the Ego4D Challenge for Long-Term Action Anticipation track at CVPR 2022