Ultrasound-based Silent Speech Interface Built on a Continuous Vocoder
Abstract
Recently it was shown that within the Silent Speech Interface (SSI) field, the prediction of F0 is possible from Ultrasound Tongue Images (UTI) as the articulatory input, using Deep Neural Networks for articulatory-to-acoustic mapping. Moreover, text-to-speech synthesizers were shown to produce higher quality speech when using a continuous pitch estimate, which takes non-zero pitch values even when voicing is not present. Therefore, in this paper on UTI-based SSI, we use a simple continuous F0 tracker which does not apply a strict voiced / unvoiced decision. Continuous vocoder parameters (ContF0, Maximum Voiced Frequency and Mel-Generalized Cepstrum) are predicted using a convolutional neural network, with UTI as input. The results demonstrate that during the articulatory-to-acoustic mapping experiments, the continuous F0 is predicted with lower error, and the continuous vocoder produces slightly more natural synthesized speech than the baseline vocoder using standard discontinuous F0.
Cite
@article{arxiv.1906.09885,
title = {Ultrasound-based Silent Speech Interface Built on a Continuous Vocoder},
author = {Tamás Gábor Csapó and Mohammed Salah Al-Radhi and Géza Németh and Gábor Gosztolya and Tamás Grósz and László Tóth and Alexandra Markó},
journal= {arXiv preprint arXiv:1906.09885},
year = {2020}
}
Comments
5 pages, 3 figures, accepted for publication at Interspeech 2019