Video Retrieval is a challenging task where a text query is matched to a video or vice versa. Most of the existing approaches for addressing such a problem rely on annotations made by the users. Although simple, this approach is not always feasible in practice. In this work, we explore the application of the language-image model, CLIP, to obtain video representations without the need for said annotations. This model was explicitly trained to learn a common space where images and text can be compared. Using various techniques described in this document, we extended its application to videos, obtaining state-of-the-art results on the MSR-VTT and MSVD benchmarks.
@article{arxiv.2102.12443,
title = {A Straightforward Framework For Video Retrieval Using CLIP},
author = {Jesús Andrés Portillo-Quintero and José Carlos Ortiz-Bayliss and Hugo Terashima-Marín},
journal= {arXiv preprint arXiv:2102.12443},
year = {2021}
}
Comments
10 pages, 1 figure, submitted to Mexican Conference for Pattern Recognition (MCPR 2021); corrected results section and added model specifications