Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering
Computation and Language
2023-05-19 v1 Audio and Speech Processing
Abstract
Self-supervised speech representation models have succeeded in various tasks, but improving them for content-related problems using unlabeled data is challenging. We propose speaker-invariant clustering (Spin), a novel self-supervised learning method that clusters speech representations and performs swapped prediction between the original and speaker-perturbed utterances. Spin disentangles speaker information and preserves content representations with just 45 minutes of fine-tuning on a single GPU. Spin improves pre-trained networks and outperforms prior methods in speech recognition and acoustic unit discovery.
Cite
@article{arxiv.2305.11072,
title = {Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering},
author = {Heng-Jui Chang and Alexander H. Liu and James Glass},
journal= {arXiv preprint arXiv:2305.11072},
year = {2023}
}
Comments
Accepted to Interspeech 2023