Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition

Priyabrata Karmakar; Shyh Wei Teng; Guojun Lu

Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition

Sound 2021-02-16 v1 Computation and Language Audio and Speech Processing

Authors: Priyabrata Karmakar , Shyh Wei Teng , Guojun Lu

Abstract

Attention is a very popular and effective mechanism in artificial neural network-based sequence-to-sequence models. In this survey paper, a comprehensive review of the different attention models used in developing automatic speech recognition systems is provided. The paper focuses on the development and evolution of attention models for offline and streaming speech recognition within recurrent neural network- and Transformer- based architectures.

Keywords

attention automatic speech recognition neural networks sequence-to-sequence transformer recurrent neural network

Cite

@article{arxiv.2102.07259,
  title  = {Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition},
  author = {Priyabrata Karmakar and Shyh Wei Teng and Guojun Lu},
  journal= {arXiv preprint arXiv:2102.07259},
  year   = {2021}
}

Comments

Submitted to IEEE/ACM Trans. on Audio, Speech, and Language Processing

Thank you for Attention: A survey on Attention-based Artificial Neural Networks for Automatic Speech Recognition

Abstract

Keywords

Cite

Comments

Related papers