Conversational Speech Naturalness Predictor
Abstract
Evaluation of conversational naturalness is essential for developing human-like speech agents. However, existing speech naturalness predictors are often designed to assess utterances from a single speaker, failing to capture conversation-level naturalness qualities. In this paper, we present a framework for an automatic naturalness predictor for two-speaker, multi-turn conversations. We first show that existing naturalness estimators have low, or sometimes even negative, correlations with conversational naturalness, based on conversational recordings annotated with human ratings. We then propose a dual-channel naturalness estimator, in which we investigate multiple pre-trained encoders with data augmentation. Our proposed model achieves substantially higher correlation with human judgments compared to existing naturalness predictors for both in-domain and out-of-domain conditions.
Cite
@article{arxiv.2603.01467,
title = {Conversational Speech Naturalness Predictor},
author = {Anfeng Xu and Yashesh Gaur and Naoyuki Kanda and Zhicheng Ouyang and Katerina Zmolikova and Desh Raj and Simone Merello and Anna Sun and Ozlem Kalinli},
journal= {arXiv preprint arXiv:2603.01467},
year = {2026}
}
Comments
Under review for Interspeech 2026