Large amounts of data has made neural machine translation (NMT) a big success in recent years. But it is still a challenge if we train these models on small-scale corpora. In this case, the way of using data appears to be more important. Here, we investigate the effective use of training data for low-resource NMT. In particular, we propose a dynamic curriculum learning (DCL) method to reorder training samples in training. Unlike previous work, we do not use a static scoring function for reordering. Instead, the order of training samples is dynamically determined in two ways - loss decline and model competence. This eases training by highlighting easy samples that the current model has enough competence to learn. We test our DCL method in a Transformer-based system. Experimental results show that DCL outperforms several strong baselines on three low-resource machine translation benchmarks and different sized data of WMT' 16 En-De.
@article{arxiv.2011.14608,
title = {Dynamic Curriculum Learning for Low-Resource Neural Machine Translation},
author = {Chen Xu and Bojie Hu and Yufan Jiang and Kai Feng and Zeyang Wang and Shen Huang and Qi Ju and Tong Xiao and Jingbo Zhu},
journal= {arXiv preprint arXiv:2011.14608},
year = {2020}
}