Video Ladder Networks
Abstract
We present the Video Ladder Network (VLN) for efficiently generating future video frames. VLN is a neural encoder-decoder model augmented at all layers by both recurrent and feedforward lateral connections. At each layer, these connections form a lateral recurrent residual block, where the feedforward connection represents a skip connection and the recurrent connection represents the residual. Thanks to the recurrent connections, the decoder can exploit temporal summaries generated from all layers of the encoder. This way, the top layer is relieved from the pressure of modeling lower-level spatial and temporal details. Furthermore, we extend the basic version of VLN to incorporate ResNet-style residual blocks in the encoder and decoder, which help improving the prediction results. VLN is trained in self-supervised regime on the Moving MNIST dataset, achieving competitive results while having very simple structure and providing fast inference.
Keywords
Cite
@article{arxiv.1612.01756,
title = {Video Ladder Networks},
author = {Francesco Cricri and Xingyang Ni and Mikko Honkala and Emre Aksu and Moncef Gabbouj},
journal= {arXiv preprint arXiv:1612.01756},
year = {2017}
}
Comments
This version extends the paper accepted at the NIPS 2016 workshop on ML for Spatiotemporal Forecasting, with more details and more experimental results