This paper describes the language identification and multilingual speech recognition system developed at Tallinn University of Technology for the Interspeech 2025 ML-SUPERB 2.0 Challenge. A hybrid language identification system is used, consisting of a pretrained language embedding model and a light-weight speech recognition model with a shared encoder across languages and language-specific bigram language models. For speech recognition, three models are used, where only a single model is applied for each language, depending on the training data availability and performance on held-out data. The model set consists of a finetuned version of SeamlessM4T, MMS-1B-all with custom language adapters and MMS-zeroshot. The system obtained the top overall score in the challenge.
@article{arxiv.2506.01458,
title = {TalTech Systems for the Interspeech 2025 ML-SUPERB 2.0 Challenge},
author = {Tanel Alumäe and Artem Fedorchenko},
journal= {arXiv preprint arXiv:2506.01458},
year = {2025}
}