Speaker-Text Retrieval via Contrastive Learning
Sound
2023-12-12 v1 Audio and Speech Processing
Abstract
In this study, we introduce a novel cross-modal retrieval task involving speaker descriptions and their corresponding audio samples. Utilizing pre-trained speaker and text encoders, we present a simple learning framework based on contrastive learning. Additionally, we explore the impact of incorporating speaker labels into the training process. Our findings establish the effectiveness of linking speaker and text information for the task for both English and Japanese languages, across diverse data configurations. Additional visual analysis unveils potential nuanced associations between speaker clustering and retrieval performance.
Cite
@article{arxiv.2312.06055,
title = {Speaker-Text Retrieval via Contrastive Learning},
author = {Xuechen Liu and Xin Wang and Erica Cooper and Xiaoxiao Miao and Junichi Yamagishi},
journal= {arXiv preprint arXiv:2312.06055},
year = {2023}
}
Comments
Submitted to IEEE Signal Processing Letters