English

Multilingual Prosody Transfer: Comparing Supervised & Transfer Learning

Computation and Language 2024-06-19 v2 Sound Audio and Speech Processing

Abstract

The field of prosody transfer in speech synthesis systems is rapidly advancing. This research is focused on evaluating learning methods for adapting pre-trained monolingual text-to-speech (TTS) models to multilingual conditions, i.e., Supervised Fine-Tuning (SFT) and Transfer Learning (TL). This comparison utilizes three distinct metrics: Mean Opinion Score (MOS), Recognition Accuracy (RA), and Mel Cepstral Distortion (MCD). Results demonstrate that, in comparison to SFT, TL leads to significantly enhanced performance, with an average MOS higher by 1.53 points, a 37.5% increase in RA, and approximately a 7.8-point improvement in MCD. These findings are instrumental in helping build TTS models for low-resource languages.

Keywords

Cite

@article{arxiv.2406.00022,
  title  = {Multilingual Prosody Transfer: Comparing Supervised & Transfer Learning},
  author = {Arnav Goel and Medha Hira and Anubha Gupta},
  journal= {arXiv preprint arXiv:2406.00022},
  year   = {2024}
}

Comments

7 pages, Accepted to ICLR 2024 - Tiny Track

R2 v1 2026-06-28T16:48:52.911Z