Accurate and Reliable Confidence Estimation Based on Non-Autoregressive End-to-End Speech Recognition System
Abstract
Estimating confidence scores for recognition results is a classic task in ASR field and of vital importance for kinds of downstream tasks and training strategies. Previous end-to-end~(E2E) based confidence estimation models (CEM) predict score sequences of equal length with input transcriptions, leading to unreliable estimation when deletion and insertion errors occur. In this paper we proposed CIF-Aligned confidence estimation model (CA-CEM) to achieve accurate and reliable confidence estimation based on novel non-autoregressive E2E ASR model - Paraformer. CA-CEM utilizes the modeling character of continuous integrate-and-fire (CIF) mechanism to generate token-synchronous acoustic embedding, which solves the estimation failure issue above. We measure the quality of estimation with AUC and RMSE in token level and ECE-U - a proposed metrics in utterance level. CA-CEM gains 24% and 19% relative reduction on ECE-U and also better AUC and RMSE on two test sets. Furthermore, we conduct analysis to explore the potential of CEM for different ASR related usage.
Keywords
Cite
@article{arxiv.2305.10680,
title = {Accurate and Reliable Confidence Estimation Based on Non-Autoregressive End-to-End Speech Recognition System},
author = {Xian Shi and Haoneng Luo and Zhifu Gao and Shiliang Zhang and Zhijie Yan},
journal= {arXiv preprint arXiv:2305.10680},
year = {2023}
}
Comments
5 pages, 4 figures, Interspeech2023