Simplified Long Short-term Memory Recurrent Neural Networks: part II
Abstract
This is part II of three-part work. Here, we present a second set of inter-related five variants of simplified Long Short-term Memory (LSTM) recurrent neural networks by further reducing adaptive parameters. Two of these models have been introduced in part I of this work. We evaluate and verify our model variants on the benchmark MNIST dataset and assert that these models are comparable to the base LSTM model while use progressively less number of parameters. Moreover, we observe that in case of using the ReLU activation, the test accuracy performance of the standard LSTM will drop after a number of epochs when learning parameter become larger. However all of the new model variants sustain their performance.
Cite
@article{arxiv.1707.04623,
title = {Simplified Long Short-term Memory Recurrent Neural Networks: part II},
author = {Atra Akandeh and Fathi M. Salem},
journal= {arXiv preprint arXiv:1707.04623},
year = {2017}
}
Comments
4 pages, 6 figures, 5 tables; this is part II of three-part work, all to appear in IKE'17- The 16th Int'l Conference on Information & Knowledge Engineering, in The 2017 World Congress in Computer Science Computer Engineering & Applied Computing | CSCE'17 July 17-20, 2017, Las Vegas, Nevada, USA