Confidence Score Based Conformer Speaker Adaptation for Speech Recognition
Abstract
A key challenge for automatic speech recognition (ASR) systems is to model the speaker level variability. In this paper, compact speaker dependent learning hidden unit contributions (LHUC) are used to facilitate both speaker adaptive training (SAT) and test time unsupervised speaker adaptation for state-of-the-art Conformer based end-to-end ASR systems. The sensitivity during adaptation to supervision error rate is reduced using confidence score based selection of the more "trustworthy" subset of speaker specific data. A confidence estimation module is used to smooth the over-confident Conformer decoder output probabilities before serving as confidence scores. The increased data sparsity due to speaker level data selection is addressed using Bayesian estimation of LHUC parameters. Experiments on the 300-hour Switchboard corpus suggest that the proposed LHUC-SAT Conformer with confidence score based test time unsupervised adaptation outperformed the baseline speaker independent and i-vector adapted Conformer systems by up to 1.0%, 1.0%, and 1.2% absolute (9.0%, 7.9%, and 8.9% relative) word error rate (WER) reductions on the NIST Hub5'00, RT02, and RT03 evaluation sets respectively. Consistent performance improvements were retained after external Transformer and LSTM language models were used for rescoring.
Keywords
Cite
@article{arxiv.2206.12045,
title = {Confidence Score Based Conformer Speaker Adaptation for Speech Recognition},
author = {Jiajun Deng and Xurong Xie and Tianzi Wang and Mingyu Cui and Boyang Xue and Zengrui Jin and Mengzhe Geng and Guinan Li and Xunying Liu and Helen Meng},
journal= {arXiv preprint arXiv:2206.12045},
year = {2022}
}
Comments
It's accepted to INTERSPEECH 2022. arXiv admin note: text overlap with arXiv:2206.11596