A recurrent neural network without chaos
Neural and Evolutionary Computing
2016-12-20 v1 Computation and Language
Machine Learning
Abstract
We introduce an exceptionally simple gated recurrent neural network (RNN) that achieves performance comparable to well-known gated architectures, such as LSTMs and GRUs, on the word-level language modeling task. We prove that our model has simple, predicable and non-chaotic dynamics. This stands in stark contrast to more standard gated architectures, whose underlying dynamical systems exhibit chaotic behavior.
Cite
@article{arxiv.1612.06212,
title = {A recurrent neural network without chaos},
author = {Thomas Laurent and James von Brecht},
journal= {arXiv preprint arXiv:1612.06212},
year = {2016}
}