Diverse Embedding Neural Network Language Models
Computation and Language
2015-04-17 v5 Machine Learning
Neural and Evolutionary Computing
Abstract
We propose Diverse Embedding Neural Network (DENN), a novel architecture for language models (LMs). A DENNLM projects the input word history vector onto multiple diverse low-dimensional sub-spaces instead of a single higher-dimensional sub-space as in conventional feed-forward neural network LMs. We encourage these sub-spaces to be diverse during network training through an augmented loss function. Our language modeling experiments on the Penn Treebank data set show the performance benefit of using a DENNLM.
Cite
@article{arxiv.1412.7063,
title = {Diverse Embedding Neural Network Language Models},
author = {Kartik Audhkhasi and Abhinav Sethy and Bhuvana Ramabhadran},
journal= {arXiv preprint arXiv:1412.7063},
year = {2015}
}
Comments
Under review as workshop contribution at ICLR 2015