Recent studies have shown that the underlying neural mechanisms of human speech comprehension can be analyzed using a match-mismatch classification of the speech stimulus and the neural response. However, such studies have been conducted for fixed-duration segments without accounting for the discrete processing of speech in the brain. In this work, we establish that word boundary information plays a significant role in sentence processing by relating EEG to its speech input. We process the speech and the EEG signals using a network of convolution layers. Then, a word boundary-based average pooling is performed on the representations, and the inter-word context is incorporated using a recurrent layer. The experiments show that the modeling accuracy can be significantly improved (match-mismatch classification accuracy) to 93% on a publicly available speech-EEG data set, while previous efforts achieved an accuracy of 65-75% for this task.
@article{arxiv.2307.00366,
title = {Enhancing the EEG Speech Match Mismatch Tasks With Word Boundaries},
author = {Akshara Soman and Vidhi Sinha and Sriram Ganapathy},
journal= {arXiv preprint arXiv:2307.00366},
year = {2023}
}
Comments
5 pages, 4 figures, 4 tables, accepted to Interspeech2023 conference