Learning Video-Story Composition via Recurrent Neural Network
Computer Vision and Pattern Recognition
2018-02-01 v1
Abstract
In this paper, we propose a learning-based method to compose a video-story from a group of video clips that describe an activity or experience. We learn the coherence between video clips from real videos via the Recurrent Neural Network (RNN) that jointly incorporates the spatial-temporal semantics and motion dynamics to generate smooth and relevant compositions. We further rearrange the results generated by the RNN to make the overall video-story compatible with the storyline structure via a submodular ranking optimization process. Experimental results on the video-story dataset show that the proposed algorithm outperforms the state-of-the-art approach.
Cite
@article{arxiv.1801.10281,
title = {Learning Video-Story Composition via Recurrent Neural Network},
author = {Guangyu Zhong and Yi-Hsuan Tsai and Sifei Liu and Zhixun Su and Ming-Hsuan Yang},
journal= {arXiv preprint arXiv:1801.10281},
year = {2018}
}