Any-to-One Sequence-to-Sequence Voice Conversion using Self-Supervised Discrete Speech Representations

Wen-Chin Huang; Yi-Chiao Wu; Tomoki Hayashi; Tomoki Toda

Any-to-One Sequence-to-Sequence Voice Conversion using Self-Supervised Discrete Speech Representations

Audio and Speech Processing 2020-10-26 v1 Computation and Language Sound

Authors: Wen-Chin Huang , Yi-Chiao Wu , Tomoki Hayashi , Tomoki Toda

Abstract

We present a novel approach to any-to-one (A2O) voice conversion (VC) in a sequence-to-sequence (seq2seq) framework. A2O VC aims to convert any speaker, including those unseen during training, to a fixed target speaker. We utilize vq-wav2vec (VQW2V), a discretized self-supervised speech representation that was learned from massive unlabeled data, which is assumed to be speaker-independent and well corresponds to underlying linguistic contents. Given a training dataset of the target speaker, we extract VQW2V and acoustic features to estimate a seq2seq mapping function from the former to the latter. With the help of a pretraining method and a newly designed postprocessing technique, our model can be generalized to only 5 min of data, even outperforming the same model trained with parallel data.

Keywords

speech processing self-supervised speech learning speaker recognition and verification

Cite

@article{arxiv.2010.12231,
  title  = {Any-to-One Sequence-to-Sequence Voice Conversion using Self-Supervised Discrete Speech Representations},
  author = {Wen-Chin Huang and Yi-Chiao Wu and Tomoki Hayashi and Tomoki Toda},
  journal= {arXiv preprint arXiv:2010.12231},
  year   = {2020}
}

Comments

Submitted to ICASSP 2021

Any-to-One Sequence-to-Sequence Voice Conversion using Self-Supervised Discrete Speech Representations

Abstract

Keywords

Cite

Comments

Related papers