Does Video Summarization Require Videos? Quantifying the Effectiveness of Language in Video Summarization
Abstract
Video summarization remains a huge challenge in computer vision due to the size of the input videos to be summarized. We propose an efficient, language-only video summarizer that achieves competitive accuracy with high data efficiency. Using only textual captions obtained via a zero-shot approach, we train a language transformer model and forego image representations. This method allows us to perform filtration amongst the representative text vectors and condense the sequence. With our approach, we gain explainability with natural language that comes easily for human interpretation and textual summaries of the videos. An ablation study that focuses on modality and data compression shows that leveraging text modality only effectively reduces input data processing while retaining comparable results.
Cite
@article{arxiv.2309.09405,
title = {Does Video Summarization Require Videos? Quantifying the Effectiveness of Language in Video Summarization},
author = {Yoonsoo Nam and Adam Lehavi and Daniel Yang and Digbalay Bose and Swabha Swayamdipta and Shrikanth Narayanan},
journal= {arXiv preprint arXiv:2309.09405},
year = {2023}
}
Comments
\c{opyright} 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works