An Empirical Analysis of Deep Audio-Visual Models for Speech Recognition
Computer Vision and Pattern Recognition
2018-12-27 v1 Computation and Language
Abstract
In this project, we worked on speech recognition, specifically predicting individual words based on both the video frames and audio. Empowered by convolutional neural networks, the recent speech recognition and lip reading models are comparable to human level performance. We re-implemented and made derivations of the state-of-the-art model. Then, we conducted rich experiments including the effectiveness of attention mechanism, more accurate residual network as the backbone with pre-trained weights and the sensitivity of our model with respect to audio input with/without noise.
Cite
@article{arxiv.1812.09336,
title = {An Empirical Analysis of Deep Audio-Visual Models for Speech Recognition},
author = {Devesh Walawalkar and Yihui He and Rohit Pillai},
journal= {arXiv preprint arXiv:1812.09336},
year = {2018}
}