Sample Efficient Adaptive Text-to-Speech
Abstract
We present a meta-learning approach for adaptive text-to-speech (TTS) with few data. During training, we learn a multi-speaker model using a shared conditional WaveNet core and independent learned embeddings for each speaker. The aim of training is not to produce a neural network with fixed weights, which is then deployed as a TTS system. Instead, the aim is to produce a network that requires few data at deployment time to rapidly adapt to new speakers. We introduce and benchmark three strategies: (i) learning the speaker embedding while keeping the WaveNet core fixed, (ii) fine-tuning the entire architecture with stochastic gradient descent, and (iii) predicting the speaker embedding with a trained neural network encoder. The experiments show that these approaches are successful at adapting the multi-speaker neural network to new speakers, obtaining state-of-the-art results in both sample naturalness and voice similarity with merely a few minutes of audio data from new speakers.
Cite
@article{arxiv.1809.10460,
title = {Sample Efficient Adaptive Text-to-Speech},
author = {Yutian Chen and Yannis Assael and Brendan Shillingford and David Budden and Scott Reed and Heiga Zen and Quan Wang and Luis C. Cobo and Andrew Trask and Ben Laurie and Caglar Gulcehre and Aäron van den Oord and Oriol Vinyals and Nando de Freitas},
journal= {arXiv preprint arXiv:1809.10460},
year = {2019}
}
Comments
Accepted by ICLR 2019