Preservation of Language Understanding Capabilities in Speech-aware Large Language Models

Marek Kubis; Paweł Skórzewski; Iwona Christop; Mateusz Czyżnikiewicz; Jakub Kubiak; Łukasz Bondaruk; Marcin Lewandowski

Preservation of Language Understanding Capabilities in Speech-aware Large Language Models

Computation and Language 2025-10-17 v2 Artificial Intelligence

Authors: Marek Kubis , Paweł Skórzewski , Iwona Christop , Mateusz Czyżnikiewicz , Jakub Kubiak , Łukasz Bondaruk , Marcin Lewandowski

View on arXiv ↗ PDF ↗

Abstract

The paper presents C3T (Cross-modal Capabilities Conservation Test), a new benchmark for assessing the performance of speech-aware large language models. The benchmark utilizes textual tasks and a voice cloning text-to-speech model to quantify the extent to which language understanding capabilities are preserved when the model is accessed via speech input. C3T quantifies the fairness of the model for different categories of speakers and its robustness across text and speech modalities.

Keywords

speech-aware large language models language understanding cross-modal benchmark voice cloning model robustness

Cite

@article{arxiv.2509.12171,
  title  = {Preservation of Language Understanding Capabilities in Speech-aware Large Language Models},
  author = {Marek Kubis and Paweł Skórzewski and Iwona Christop and Mateusz Czyżnikiewicz and Jakub Kubiak and Łukasz Bondaruk and Marcin Lewandowski},
  journal= {arXiv preprint arXiv:2509.12171},
  year   = {2025}
}

Comments

5 pages, 1 figure; benchmark code available at https://github.com/SamsungLabs/C3T

Preservation of Language Understanding Capabilities in Speech-aware Large Language Models

Abstract

Keywords

Cite

Comments

Related papers