Natural Language Processing with Small Feed-Forward Networks
Computation and Language
2017-08-02 v1 Neural and Evolutionary Computing
Abstract
We show that small and shallow feed-forward neural networks can achieve near state-of-the-art results on a range of unstructured and structured language processing tasks while being considerably cheaper in memory and computational requirements than deep recurrent models. Motivated by resource-constrained environments like mobile phones, we showcase simple techniques for obtaining such small neural network models, and investigate different tradeoffs when deciding how to allocate a small memory budget.
Cite
@article{arxiv.1708.00214,
title = {Natural Language Processing with Small Feed-Forward Networks},
author = {Jan A. Botha and Emily Pitler and Ji Ma and Anton Bakalov and Alex Salcianu and David Weiss and Ryan McDonald and Slav Petrov},
journal= {arXiv preprint arXiv:1708.00214},
year = {2017}
}
Comments
EMNLP 2017 short paper