Multilingual speech recognition for both monolingual and code-switching speech is a challenging task. Recently, based on the Mixture of Experts (MoE), many works have made good progress in multilingual and code-switching ASR, but present huge computational complexity with the increase of supported languages. In this work, we propose a computation-efficient network named Language-Routing Mixture of Experts (LR-MoE) for multilingual and code-switching ASR. LR-MoE extracts language-specific representations through the Mixture of Language Experts (MLE), which is guided to learn by a frame-wise language routing mechanism. The weight-shared frame-level language identification (LID) network is jointly trained as the shared pre-router of each MoE layer. Experiments show that the proposed method significantly improves multilingual and code-switching speech recognition performances over baseline with comparable computational efficiency.
@article{arxiv.2307.05956,
title = {Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition},
author = {Wenxuan Wang and Guodong Ma and Yuke Li and Binbin Du},
journal= {arXiv preprint arXiv:2307.05956},
year = {2023}
}
Comments
To appear in Proc. INTERSPEECH 2023, August 20-24, 2023, Dublin, Ireland