Attention-based Neural Beamforming Layers for Multi-channel Speech Recognition
Audio and Speech Processing
2021-05-18 v2 Machine Learning
Sound
Abstract
Attention-based beamformers have recently been shown to be effective for multi-channel speech recognition. However, they are less capable at capturing local information. In this work, we propose a 2D Conv-Attention module which combines convolution neural networks with attention for beamforming. We apply self- and cross-attention to explicitly model the correlations within and between the input channels. The end-to-end 2D Conv-Attention model is compared with a multi-head self-attention and superdirective-based neural beamformers. We train and evaluate on an in-house multi-channel dataset. The results show a relative improvement of 3.8% in WER by the proposed model over the baseline neural beamformer.
Cite
@article{arxiv.2105.05920,
title = {Attention-based Neural Beamforming Layers for Multi-channel Speech Recognition},
author = {Bhargav Pulugundla and Yang Gao and Brian King and Gokce Keskin and Harish Mallidi and Minhua Wu and Jasha Droppo and Roland Maas},
journal= {arXiv preprint arXiv:2105.05920},
year = {2021}
}