Expressive paragraph text-to-speech synthesis with multi-step variational autoencoder
Abstract
Neural networks have been able to generate high-quality single-sentence speech. However, it remains a challenge concerning audio-book speech synthesis due to the intra-paragraph correlation of semantic and acoustic features as well as variable styles. In this paper, we propose a highly expressive paragraph speech synthesis system with a multi-step variational autoencoder, called EP-MSTTS. EP-MSTTS is the first VITS-based paragraph speech synthesis model and models the variable style of paragraph speech at five levels: frame, phoneme, word, sentence, and paragraph. We also propose a series of improvements to enhance the performance of this hierarchical model. In addition, we directly train EP-MSTTS on speech sliced by paragraph rather than sentence. Experiment results on the single-speaker French audiobook corpus released at Blizzard Challenge 2023 show EP-MSTTS obtains better performance than baseline models.
Keywords
Cite
@article{arxiv.2308.13365,
title = {Expressive paragraph text-to-speech synthesis with multi-step variational autoencoder},
author = {Xuyuan Li and Zengqiang Shang and Peiyang Shi and Hua Hua and Ta Li and Pengyuan Zhang},
journal= {arXiv preprint arXiv:2308.13365},
year = {2024}
}
Comments
accepted at Interspeech 2024