Automatic Song Translation for Tonal Languages
Computation and Language
2022-03-28 v1 Artificial Intelligence
Sound
Audio and Speech Processing
Abstract
This paper develops automatic song translation (AST) for tonal languages and addresses the unique challenge of aligning words' tones with melody of a song in addition to conveying the original meaning. We propose three criteria for effective AST -- preserving meaning, singability and intelligibility -- and design metrics for these criteria. We develop a new benchmark for English--Mandarin song translation and develop an unsupervised AST system, Guided AliGnment for Automatic Song Translation (GagaST), which combines pre-training with three decoding constraints. Both automatic and human evaluations show GagaST successfully balances semantics and singability.
Keywords
Cite
@article{arxiv.2203.13420,
title = {Automatic Song Translation for Tonal Languages},
author = {Fenfei Guo and Chen Zhang and Zhirui Zhang and Qixin He and Kejun Zhang and Jun Xie and Jordan Boyd-Graber},
journal= {arXiv preprint arXiv:2203.13420},
year = {2022}
}
Comments
Accepted at Findings of ACL 2022, 15 pages, 4 Tables and 10 Figures