Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect
Abstract
Perceptual voice quality dimensions describe key characteristics of atypical speech and other speech modulations. Here we develop and evaluate voice quality models for seven voice and speech dimensions (intelligibility, imprecise consonants, harsh voice, naturalness, monoloudness, monopitch, and breathiness). Probes were trained on the public Speech Accessibility (SAP) project dataset with 11,184 samples from 434 speakers, using embeddings from frozen pre-trained models as features. We found that our probes had both strong performance and strong generalization across speech elicitation categories in the SAP dataset. We further validated zero-shot performance on additional datasets, encompassing unseen languages and tasks: Italian atypical speech, English atypical speech, and affective speech. The strong zero-shot performance and the interpretability of results across an array of evaluations suggests the utility of using voice quality dimensions in speaking style-related tasks.
Keywords
Cite
@article{arxiv.2505.21809,
title = {Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect},
author = {Jaya Narain and Vasudha Kowtha and Colin Lea and Lauren Tooley and Dianna Yee and Vikramjit Mitra and Zifang Huang and Miquel Espi Marques and Jon Huang and Carlos Avendano and Shirley Ren},
journal= {arXiv preprint arXiv:2505.21809},
year = {2025}
}
Comments
accepted for Interspeech 2025