Multi-Scale Spectrogram Modelling for Neural Text-to-Speech
Abstract
We propose a novel Multi-Scale Spectrogram (MSS) modelling approach to synthesise speech with an improved coarse and fine-grained prosody. We present a generic multi-scale spectrogram prediction mechanism where the system first predicts coarser scale mel-spectrograms that capture the suprasegmental information in speech, and later uses these coarser scale mel-spectrograms to predict finer scale mel-spectrograms capturing fine-grained prosody. We present details for two specific versions of MSS called Word-level MSS and Sentence-level MSS where the scales in our system are motivated by the linguistic units. The Word-level MSS models word, phoneme, and frame-level spectrograms while Sentence-level MSS models sentence-level spectrogram in addition. Subjective evaluations show that Word-level MSS performs statistically significantly better compared to the baseline on two voices.
Cite
@article{arxiv.2106.15649,
title = {Multi-Scale Spectrogram Modelling for Neural Text-to-Speech},
author = {Ammar Abbas and Bajibabu Bollepalli and Alexis Moinet and Arnaud Joly and Penny Karanasou and Peter Makarov and Simon Slangens and Sri Karlapati and Thomas Drugman},
journal= {arXiv preprint arXiv:2106.15649},
year = {2021}
}
Comments
Accepted for the 11th ISCA Speech Synthesis Workshop (SSW11)