Causal Video Summarizer for Video Exploration
Abstract
Recently, video summarization has been proposed as a method to help video exploration. However, traditional video summarization models only generate a fixed video summary which is usually independent of user-specific needs and hence limits the effectiveness of video exploration. Multi-modal video summarization is one of the approaches utilized to address this issue. Multi-modal video summarization has a video input and a text-based query input. Hence, effective modeling of the interaction between a video input and text-based query is essential to multi-modal video summarization. In this work, a new causality-based method named Causal Video Summarizer (CVS) is proposed to effectively capture the interactive information between the video and query to tackle the task of multi-modal video summarization. The proposed method consists of a probabilistic encoder and a probabilistic decoder. Based on the evaluation of the existing multi-modal video summarization dataset, experimental results show that the proposed approach is effective with the increase of +5.4% in accuracy and +4.92% increase of F 1- score, compared with the state-of-the-art method.
Cite
@article{arxiv.2307.01947,
title = {Causal Video Summarizer for Video Exploration},
author = {Jia-Hong Huang and Chao-Han Huck Yang and Pin-Yu Chen and Andrew Brown and Marcel Worring},
journal= {arXiv preprint arXiv:2307.01947},
year = {2023}
}
Comments
This paper is accepted by IEEE International Conference on Multimedia and Expo (ICME), 2022