Learning from others' mistakes: Finetuning machine translation models with span-level error annotations
Abstract
Despite growing interest in incorporating feedback to improve language models, most efforts focus only on sequence-level annotations. In this work, we explore the potential of utilizing fine-grained span-level annotations from offline datasets to improve model quality. We develop a simple finetuning algorithm, called Training with Annotations (TWA), to directly train machine translation models on such annotated data. TWA utilizes targeted span-level error information while also flexibly learning what to penalize within a span. Moreover, TWA considers the overall trajectory of a sequence when deciding which non-error spans to utilize as positive signals. Experiments on English-German and Chinese-English machine translation show that TWA outperforms baselines such as Supervised FineTuning on sequences filtered for quality and Direct Preference Optimization on pairs constructed from the same data.
Cite
@article{arxiv.2410.16509,
title = {Learning from others' mistakes: Finetuning machine translation models with span-level error annotations},
author = {Lily H. Zhang and Hamid Dadkhahi and Mara Finkelstein and Firas Trabelsi and Jiaming Luo and Markus Freitag},
journal= {arXiv preprint arXiv:2410.16509},
year = {2024}
}