Generative Modeling for Low Dimensional Speech Attributes with Neural Spline Flows
Sound
2022-06-28 v4 Machine Learning
Audio and Speech Processing
Abstract
Despite recent advances in generative modeling for text-to-speech synthesis, these models do not yet have the same fine-grained adjustability of pitch-conditioned deterministic models such as FastPitch and FastSpeech2. Pitch information is not only low-dimensional, but also discontinuous, making it particularly difficult to model in a generative setting. Our work explores several techniques for handling the aforementioned issues in the context of Normalizing Flow models. We also find this problem to be very well suited for Neural Spline flows, which is a highly expressive alternative to the more common affine-coupling mechanism in Normalizing Flows.
Keywords
Cite
@article{arxiv.2203.01786,
title = {Generative Modeling for Low Dimensional Speech Attributes with Neural Spline Flows},
author = {Kevin J. Shih and Rafael Valle and Rohan Badlani and João Felipe Santos and Bryan Catanzaro},
journal= {arXiv preprint arXiv:2203.01786},
year = {2022}
}
Comments
22 pages, 11 figures, 3 tables