Layer-wise Analysis for Quality of Multilingual Synthesized Speech
Abstract
While supervised quality predictors for synthesized speech have demonstrated strong correlations with human ratings, their requirement for in-domain labeled training data hinders their generalization ability to new domains. Unsupervised approaches based on pretrained self-supervised learning (SSL) based models and automatic speech recognition (ASR) models are a promising alternative; however, little is known about how these models encode information about speech quality. Towards the goal of better understanding how different aspects of speech quality are encoded in a multilingual setting, we present a layer-wise analysis of multilingual pretrained speech models based on reference modeling. We find that features extracted from early SSL layers show correlations with human ratings of synthesized speech, and later layers of ASR models can predict quality of non-neural systems as well as intelligibility. We also demonstrate the importance of using well-matched reference data.
Keywords
Cite
@article{arxiv.2509.04830,
title = {Layer-wise Analysis for Quality of Multilingual Synthesized Speech},
author = {Erica Cooper and Takuma Okamoto and Yamato Ohtani and Tomoki Toda and Hisashi Kawai},
journal= {arXiv preprint arXiv:2509.04830},
year = {2025}
}
Comments
Copyright 2025 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works