Phoneme-aware Encoding for Prefix-tree-based Contextual ASR
Abstract
In speech recognition applications, it is important to recognize context-specific rare words, such as proper nouns. Tree-constrained Pointer Generator (TCPGen) has shown promise for this purpose, which efficiently biases such words with a prefix tree. While the original TCPGen relies on grapheme-based encoding, we propose extending it with phoneme-aware encoding to better recognize words of unusual pronunciations. As TCPGen handles biasing words as subword units, we propose obtaining subword-level phoneme-aware encoding by using alignment between phonemes and subwords. Furthermore, we propose injecting phoneme-level predictions from CTC into queries of TCPGen so that the model better interprets the phoneme-aware encodings. We conducted ASR experiments with TCPGen for RNN transducer. We observed that proposed phoneme-aware encoding outperformed ordinary grapheme-based encoding on both the English LibriSpeech and Japanese CSJ datasets, demonstrating the robustness of our approach across linguistically diverse languages.
Keywords
Cite
@article{arxiv.2312.09582,
title = {Phoneme-aware Encoding for Prefix-tree-based Contextual ASR},
author = {Hayato Futami and Emiru Tsunoo and Yosuke Kashiwagi and Hiroaki Ogawa and Siddhant Arora and Shinji Watanabe},
journal= {arXiv preprint arXiv:2312.09582},
year = {2023}
}
Comments
Accepted to ICASSP2024