Two techniques provide the fabric of the Cambridge University Engineering Department's (CUED) entry to the WMT19 evaluation campaign: elastic weight consolidation (EWC) and different forms of language modelling (LMs). We report substantial gains by fine-tuning very strong baselines on former WMT test sets using a combination of checkpoint averaging and EWC. A sentence-level Transformer LM and a document-level LM based on a modified Transformer architecture yield further gains. As in previous years, we also extract n-gram probabilities from SMT lattices which can be seen as a source-conditioned n-gram LM.
@article{arxiv.1906.05447,
title = {Cued@wmt19:ewc&lms},
author = {Felix Stahlberg and Danielle Saunders and Adria de Gispert and Bill Byrne},
journal= {arXiv preprint arXiv:1906.05447},
year = {2019}
}
Comments
WMT2019 system description (University of Cambridge)