We introduce XTREME-S, a new benchmark to evaluate universal cross-lingual speech representations in many languages. XTREME-S covers four task families: speech recognition, classification, speech-to-text translation and retrieval. Covering 102 languages from 10+ language families, 3 different domains and 4 task families, XTREME-S aims to simplify multilingual speech representation evaluation, as well as catalyze research in "universal" speech representation learning. This paper describes the new benchmark and establishes the first speech-only and speech-text baselines using XLS-R and mSLAM on all downstream tasks. We motivate the design choices and detail how to use the benchmark. Datasets and fine-tuning scripts are made easily accessible at https://hf.co/datasets/google/xtreme_s.
@article{arxiv.2203.10752,
title = {XTREME-S: Evaluating Cross-lingual Speech Representations},
author = {Alexis Conneau and Ankur Bapna and Yu Zhang and Min Ma and Patrick von Platen and Anton Lozhkov and Colin Cherry and Ye Jia and Clara Rivera and Mihir Kale and Daan Van Esch and Vera Axelrod and Simran Khanuja and Jonathan H. Clark and Orhan Firat and Michael Auli and Sebastian Ruder and Jason Riesa and Melvin Johnson},
journal= {arXiv preprint arXiv:2203.10752},
year = {2022}
}
Comments
Minor fix: language code for Filipino (Tagalog), "tg" -> "tl"