Neural Machine Translation with Recurrent Attention Modeling
Neural and Evolutionary Computing
2016-07-19 v1 Computation and Language
Abstract
Knowing which words have been attended to in previous time steps while generating a translation is a rich source of information for predicting what words will be attended to in the future. We improve upon the attention model of Bahdanau et al. (2014) by explicitly modeling the relationship between previous and subsequent attention levels for each word using one recurrent network per input word. This architecture easily captures informative features, such as fertility and regularities in relative distortion. In experiments, we show our parameterization of attention improves translation quality.
Cite
@article{arxiv.1607.05108,
title = {Neural Machine Translation with Recurrent Attention Modeling},
author = {Zichao Yang and Zhiting Hu and Yuntian Deng and Chris Dyer and Alex Smola},
journal= {arXiv preprint arXiv:1607.05108},
year = {2016}
}