Improving Conditioning in Context-Aware Sequence to Sequence Models
Computation and Language
2019-11-25 v1 Machine Learning
Abstract
Neural sequence to sequence models are well established for applications which can be cast as mapping a single input sequence into a single output sequence. In this work, we focus on cases where generation is conditioned on both a short query and a long context, such as abstractive question answering or document-level translation. We modify the standard sequence-to-sequence approach to make better use of both the query and the context by expanding the conditioning mechanism to intertwine query and context attention. We also introduce a simple and efficient data augmentation method for the proposed model. Experiments on three different tasks show that both changes lead to consistent improvements.
Cite
@article{arxiv.1911.09728,
title = {Improving Conditioning in Context-Aware Sequence to Sequence Models},
author = {Xinyi Wang and Jason Weston and Michael Auli and Yacine Jernite},
journal= {arXiv preprint arXiv:1911.09728},
year = {2019}
}