Semi-Supervised Generative Modeling for Controllable Speech Synthesis
Computation and Language
2019-10-07 v1 Machine Learning
Sound
Audio and Speech Processing
Abstract
We present a novel generative model that combines state-of-the-art neural text-to-speech (TTS) with semi-supervised probabilistic latent variable models. By providing partial supervision to some of the latent variables, we are able to force them to take on consistent and interpretable purposes, which previously hasn't been possible with purely unsupervised TTS models. We demonstrate that our model is able to reliably discover and control important but rarely labelled attributes of speech, such as affect and speaking rate, with as little as 1% (30 minutes) supervision. Even at such low supervision levels we do not observe a degradation of synthesis quality compared to a state-of-the-art baseline. Audio samples are available on the web.
Cite
@article{arxiv.1910.01709,
title = {Semi-Supervised Generative Modeling for Controllable Speech Synthesis},
author = {Raza Habib and Soroosh Mariooryad and Matt Shannon and Eric Battenberg and RJ Skerry-Ryan and Daisy Stanton and David Kao and Tom Bagby},
journal= {arXiv preprint arXiv:1910.01709},
year = {2019}
}