Improving the Transformer Translation Model with Document-Level Context
Abstract
Although the Transformer translation model (Vaswani et al., 2017) has achieved state-of-the-art performance in a variety of translation tasks, how to use document-level context to deal with discourse phenomena problematic for Transformer still remains a challenge. In this work, we extend the Transformer model with a new context encoder to represent document-level context, which is then incorporated into the original encoder and decoder. As large-scale document-level parallel corpora are usually not available, we introduce a two-step training method to take full advantage of abundant sentence-level parallel corpora and limited document-level parallel corpora. Experiments on the NIST Chinese-English datasets and the IWSLT French-English datasets show that our approach improves over Transformer significantly.
Cite
@article{arxiv.1810.03581,
title = {Improving the Transformer Translation Model with Document-Level Context},
author = {Jiacheng Zhang and Huanbo Luan and Maosong Sun and FeiFei Zhai and Jingfang Xu and Min Zhang and Yang Liu},
journal= {arXiv preprint arXiv:1810.03581},
year = {2018}
}
Comments
EMNLP 2018