Transducer neural networks have emerged as the mainstream approach for streaming automatic speech recognition (ASR), offering state-of-the-art performance in balancing accuracy and latency. In the conventional framework, streaming transducer models are trained to maximize the likelihood function based on non-streaming recursion rules. However, this approach leads to a mismatch between training and inference, resulting in the issue of deformed likelihood and consequently suboptimal ASR accuracy. We introduce a mathematical quantification of the gap between the actual likelihood and the deformed likelihood, namely forward variable causal compensation (FoCC). We also present its estimator, FoCCE, as a solution to estimate the exact likelihood. Through experiments on the LibriSpeech dataset, we show that FoCCE training improves the accuracy of the streaming transducers.
@article{arxiv.2411.17537,
title = {Towards Maximum Likelihood Training for Transducer-based Streaming Speech Recognition},
author = {Hyeonseung Lee and Ji Won Yoon and Sungsoo Kim and Nam Soo Kim},
journal= {arXiv preprint arXiv:2411.17537},
year = {2024}
}