The advantage of recurrent neural networks (RNNs) in learning dependencies between time-series data has distinguished RNNs from other deep learning models. Recently, many advances are proposed in this emerging field. However, there is a lack of comprehensive review on memory models in RNNs in the literature. This paper provides a fundamental review on RNNs and long short term memory (LSTM) model. Then, provides a surveys of recent advances in different memory enhancements and learning techniques for capturing long term dependencies in RNNs.
@article{arxiv.1602.04335,
title = {Learning Over Long Time Lags},
author = {Hojjat Salehinejad},
journal= {arXiv preprint arXiv:1602.04335},
year = {2016}
}
Comments
This is a draft article, in preparation to submit for peer-review