Token-based multitasking frameworks like TokenVerse require all training utterances to have labels for all tasks, hindering their ability to leverage partially annotated datasets and scale effectively. We propose TokenVerse++, which introduces learnable vectors in the acoustic embedding space of the XLSR-Transducer ASR model for dynamic task activation. This core mechanism enables training with utterances labeled for only a subset of tasks, a key advantage over TokenVerse. We demonstrate this by successfully integrating a dataset with partial labels, specifically for ASR and an additional task, language identification, improving overall performance. TokenVerse++ achieves results on par with or exceeding TokenVerse across multiple tasks, establishing it as a more practical multitask alternative without sacrificing ASR performance.
@article{arxiv.2508.19856,
title = {TokenVerse++: Towards Flexible Multitask Learning with Dynamic Task Activation},
author = {Shashi Kumar and Srikanth Madikeri and Esaú Villatoro-Tello and Sergio Burdisso and Pradeep Rangappa and Andrés Carofilis and Petr Motlicek and Karthik Pandia and Shankar Venkatesan and Kadri Hacioğlu and Andreas Stolcke},
journal= {arXiv preprint arXiv:2508.19856},
year = {2025}
}
Comments
Accepted to IEEE ASRU 2025. Copyright\copyright 2025 IEEE