Semi-supervised multi-channel speaker diarization with cross-channel attention
Abstract
Most neural speaker diarization systems rely on sufficient manual training data labels, which are hard to collect under real-world scenarios. This paper proposes a semi-supervised speaker diarization system to utilize large-scale multi-channel training data by generating pseudo-labels for unlabeled data. Furthermore, we introduce cross-channel attention into the Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding (NSD-MA-MSE) to learn channel contextual information of speaker embeddings better. Experimental results on the CHiME-7 Mixer6 dataset which only contains partial speakers' labels of the training set, show that our system achieved 57.01% relative DER reduction compared to the clustering-based model on the development set. We further conducted experiments on the CHiME-6 dataset to simulate the scenario of missing partial training set labels. When using 80% and 50% labeled training data, our system performs comparably to the results obtained using 100% labeled data for training.
Cite
@article{arxiv.2307.08688,
title = {Semi-supervised multi-channel speaker diarization with cross-channel attention},
author = {Shilong Wu and Jun Du and Maokui He and Shutong Niu and Hang Chen and Haitao Tang and Chin-Hui Lee},
journal= {arXiv preprint arXiv:2307.08688},
year = {2023}
}
Comments
8 pages,3 figures