Joint Beamforming and Speaker-Attributed ASR for Real Distant-Microphone Meeting Transcription
Abstract
Distant-microphone meeting transcription is a challenging task. State-of-the-art end-to-end speaker-attributed automatic speech recognition (SA-ASR) architectures lack a multichannel noise and reverberation reduction front-end, which limits their performance. In this paper, we introduce a joint beamforming and SA-ASR approach for real meeting transcription. We first describe a data alignment and augmentation method to pretrain a neural beamformer on real meeting data. We then compare fixed, hybrid, and fully neural beamformers as front-ends to the SA-ASR model. Finally, we jointly optimize the fully neural beamformer and the SA-ASR model. Experiments on the real AMI corpus show that, while state-of-the-art multi-frame cross-channel attention based channel fusion fails to improve ASR performance, fine-tuning SA-ASR on the fixed beamformer's output and jointly fine-tuning SA-ASR with the neural beamformer reduce the word error rate by 8% and 9% relative, respectively.
Keywords
Cite
@article{arxiv.2410.21849,
title = {Joint Beamforming and Speaker-Attributed ASR for Real Distant-Microphone Meeting Transcription},
author = {Can Cui and Imran Ahamad Sheikh and Mostafa Sadeghi and Emmanuel Vincent},
journal= {arXiv preprint arXiv:2410.21849},
year = {2025}
}