Given recent advances in generative AI technology, a key question is how large language models (LLMs) can enhance acoustic modeling tasks using text decoding results from a frozen, pretrained automatic speech recognition (ASR) model. To explore new capabilities in language modeling for speech processing, we introduce the generative speech transcription error correction (GenSEC) challenge. This challenge comprises three post-ASR language modeling tasks: (i) post-ASR transcription correction, (ii) speaker tagging, and (iii) emotion recognition. These tasks aim to emulate future LLM-based agents handling voice-based interfaces while remaining accessible to a broad audience by utilizing open pretrained language models or agent-based APIs. We also discuss insights from baseline evaluations, as well as lessons learned for designing future evaluations.
@article{arxiv.2409.09785,
title = {Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition},
author = {Chao-Han Huck Yang and Taejin Park and Yuan Gong and Yuanchao Li and Zhehuai Chen and Yen-Ting Lin and Chen Chen and Yuchen Hu and Kunal Dhawan and Piotr Żelasko and Chao Zhang and Yun-Nung Chen and Yu Tsao and Jagadeesh Balam and Boris Ginsburg and Sabato Marco Siniscalchi and Eng Siong Chng and Peter Bell and Catherine Lai and Shinji Watanabe and Andreas Stolcke},
journal= {arXiv preprint arXiv:2409.09785},
year = {2025}
}
Comments
IEEE SLT 2024. The initial draft version has been done in December 2023. Post-ASR Text Processing and Understanding Community and LlaMA-7B pre-training correction model: https://huggingface.co/GenSEC-LLM/SLT-Task1-Llama2-7b-HyPo-baseline