English

EEG based Continuous Speech Recognition using Transformers

Audio and Speech Processing 2020-05-06 v3 Machine Learning Sound Machine Learning
Authors: Gautam Krishna , Co Tran , Mason Carnahan , Ahmed H Tewfik
View on arXiv ↗ PDF ↗

Abstract

In this paper we investigate continuous speech recognition using electroencephalography (EEG) features using recently introduced end-to-end transformer based automatic speech recognition (ASR) model. Our results demonstrate that transformer based model demonstrate faster training compared to recurrent neural network (RNN) based sequence-to-sequence EEG models and better performance during inference time for smaller test set vocabulary but as we increase the vocabulary size, the performance of the RNN based models were better than transformer based model on a limited English vocabulary.

Keywords

automatic speech recognition eeg signal processing speech and audio processing

Cite

@article{arxiv.2001.00501,
  title  = {EEG based Continuous Speech Recognition using Transformers},
  author = {Gautam Krishna and Co Tran and Mason Carnahan and Ahmed H Tewfik},
  journal= {arXiv preprint arXiv:2001.00501},
  year   = {2020}
}

Related papers

View all related →
Machine Learning · Computer Science
Continuous Speech Recognition using EEG and Video
Gautam Krishna, Mason Carnahan, Co Tran, Ahmed H Tewfik
2020-01-01
Audio and Speech Processing · Electrical Eng. & Systems
State-of-the-art Speech Recognition using EEG and Towards Decoding of Speech Spectrum From EEG
Gautam Krishna, Yan Han, Co Tran, Mason Carnahan +1
2020-03-06
Audio and Speech Processing · Electrical Eng. & Systems
Improving EEG based Continuous Speech Recognition
Gautam Krishna, Co Tran, Mason Carnahan, Yan Han +1
2019-12-25
Audio and Speech Processing · Electrical Eng. & Systems
Advancing Speech Recognition With No Speech Or With Noisy Speech
Gautam Krishna, Co Tran, Mason Carnahan, Ahmed H Tewfik
2020-03-17
Audio and Speech Processing · Electrical Eng. & Systems
Continuous Silent Speech Recognition using EEG
Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik
2020-05-06
Audio and Speech Processing · Electrical Eng. & Systems
Improving EEG based continuous speech recognition using GAN
Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik
2020-06-03
Audio and Speech Processing · Electrical Eng. & Systems
A comparison of end-to-end models for long-form speech recognition
Chung-Cheng Chiu, Wei Han, Yu Zhang, Ruoming Pang +10
2019-11-07
Audio and Speech Processing · Electrical Eng. & Systems
Constrained Variational Autoencoder for improving EEG based Speech Recognition Systems
Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik
2020-06-05
Machine Learning · Computer Science
Speech Recognition with no speech or with noisy speech
Gautam Krishna, Co Tran, Jianguo Yu, Ahmed H Tewfik
2019-03-05
Computation and Language · Computer Science
Semantic Mask for Transformer based End-to-End Speech Recognition
Chengyi Wang, Yu Wu, Yujiao Du, Jinyu Li +6
2020-03-17
Computation and Language · Computer Science
Echo State Speech Recognition
Harsh Shrivastava, Ankush Garg, Yuan Cao, Yu Zhang +1
2021-02-19
Audio and Speech Processing · Electrical Eng. & Systems
Attention-based ASR with Lightweight and Dynamic Convolutions
Yuya Fujita, Aswin Shanmugam Subramanian, Motoi Omachi, Shinji Watanabe
2020-02-21
Computation and Language · Computer Science
Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers
Takaaki Hori, Niko Moritz, Chiori Hori, Jonathan Le Roux
2021-04-20
Audio and Speech Processing · Electrical Eng. & Systems
End-to-End Multi-Speaker Speech Recognition using Speaker Embeddings and Transfer Learning
Pavel Denisov, Ngoc Thang Vu
2019-08-14
Audio and Speech Processing · Electrical Eng. & Systems
End-to-End Speech Recognition: A Survey
Rohit Prabhavalkar, Takaaki Hori, Tara N. Sainath, Ralf Schlüter +1
2023-03-07
Audio and Speech Processing · Electrical Eng. & Systems
Enhancing CTC-based speech recognition with diverse modeling units
Shiyi Han, Zhihong Lei, Mingbin Xu, Xingyu Na +1
2024-06-12
Audio and Speech Processing · Electrical Eng. & Systems
Disentangled-Transformer: An Explainable End-to-End Automatic Speech Recognition Model with Speech Content-Context Separation
Pu Wang, Hugo Van hamme
2024-11-28
Computation and Language · Computer Science
A Simplified Fully Quantized Transformer for End-to-end Speech Recognition
Alex Bie, Bharat Venkitesh, Joao Monteiro, Md. Akmal Haidar +1
2020-03-25
Audio and Speech Processing · Electrical Eng. & Systems
Speech Synthesis using EEG
Gautam Krishna, Co Tran, Yan Han, Mason Carnahan
2020-05-05
Sound · Computer Science
Unified End-to-End Speech Recognition and Endpointing for Fast and Efficient Speech Systems
Shaan Bijwadia, Shuo-yiin Chang, Bo Li, Tara Sainath +2
2023-02-16
Audio and Speech Processing · Electrical Eng. & Systems
An Investigation of End-to-End Models for Robust Speech Recognition
Archiki Prasad, Preethi Jyothi, Rajbabu Velmurugan
2021-02-15
Audio and Speech Processing · Electrical Eng. & Systems
End-to-End Multi-speaker Speech Recognition with Transformer
Xuankai Chang, Wangyou Zhang, Yanmin Qian, Jonathan Le Roux +1
2020-02-14
Computation and Language · Computer Science
End-to-end contextual speech recognition using class language models and a token passing decoder
Zhehuai Chen, Mahaveer Jain, Yongqiang Wang, Michael L. Seltzer +1
2018-12-06
Audio and Speech Processing · Electrical Eng. & Systems
Relaxed Attention: A Simple Method to Boost Performance of End-to-End Automatic Speech Recognition
Timo Lohrenz, Patrick Schwarz, Zhengyang Li, Tim Fingscheidt
2021-12-16
Computation and Language · Computer Science
ASR-Aware End-to-end Neural Diarization
Aparna Khare, Eunjung Han, Yuguang Yang, Andreas Stolcke
2022-07-13