Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion
Abstract
Voice Activity Detection (VAD) plays a key role in speech processing, often utilizing hand-crafted or neural features. This study examines the effectiveness of Mel-Frequency Cepstral Coefficients (MFCCs) and pre-trained model (PTM) features, including wav2vec 2.0, HuBERT, WavLM, UniSpeech, MMS, and Whisper. We propose FusionVAD, a unified framework that combines both feature types using three fusion strategies: concatenation, addition, and cross-attention (CA). Experimental results reveal that simple fusion techniques, particularly addition, outperform CA in both accuracy and efficiency. Fusion-based models consistently surpass single-feature models, highlighting the complementary nature of MFCCs and PTM features. Notably, our best-performing fusion model exceeds the state-of-the-art Pyannote across multiple datasets, achieving an absolute average improvement of 2.04%. These results confirm that simple feature fusion enhances VAD robustness while maintaining computational efficiency.
Cite
@article{arxiv.2506.01365,
title = {Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion},
author = {Kumud Tripathi and Chowdam Venkata Kumar and Pankaj Wasnik},
journal= {arXiv preprint arXiv:2506.01365},
year = {2025}
}
Comments
Accepted at INTERSPEECH 2025, 5 pages, 4 figures, 2 tables