We address the problem of end-to-end visual storytelling. Given a photo album, our model first selects the most representative (summary) photos, and then composes a natural language story for the album. For this task, we make use of the Visual Storytelling dataset and a model composed of three hierarchically-attentive Recurrent Neural Nets (RNNs) to: encode the album photos, select representative (summary) photos, and compose the story. Automatic and human evaluations show our model achieves better performance on selection, generation, and retrieval than baselines.
@article{arxiv.1708.02977,
title = {Hierarchically-Attentive RNN for Album Summarization and Storytelling},
author = {Licheng Yu and Mohit Bansal and Tamara L. Berg},
journal= {arXiv preprint arXiv:1708.02977},
year = {2017}
}