DreamVoice: Text-Guided Voice Conversion
Abstract
Generative voice technologies are rapidly evolving, offering opportunities for more personalized and inclusive experiences. Traditional one-shot voice conversion (VC) requires a target recording during inference, limiting ease of usage in generating desired voice timbres. Text-guided generation offers an intuitive solution to convert voices to desired "DreamVoices" according to the users' needs. Our paper presents two major contributions to VC technology: (1) DreamVoiceDB, a robust dataset of voice timbre annotations for 900 speakers from VCTK and LibriTTS. (2) Two text-guided VC methods: DreamVC, an end-to-end diffusion-based text-guided VC model; and DreamVG, a versatile text-to-voice generation plugin that can be combined with any one-shot VC models. The experimental results demonstrate that our proposed methods trained on the DreamVoiceDB dataset generate voice timbres accurately aligned with the text prompt and achieve high-quality VC.
Keywords
Cite
@article{arxiv.2406.16314,
title = {DreamVoice: Text-Guided Voice Conversion},
author = {Jiarui Hai and Karan Thakkar and Helin Wang and Zengyi Qin and Mounya Elhilali},
journal= {arXiv preprint arXiv:2406.16314},
year = {2024}
}
Comments
Accepted at INTERSPEECH 2024