Active Learning Based Fine-Tuning Framework for Speech Emotion Recognition
Abstract
Speech emotion recognition (SER) has drawn increasing attention for its applications in human-machine interaction. However, existing SER methods ignore the information gap between the pre-training speech recognition task and the downstream SER task, leading to sub-optimal performance. Moreover, they require much time to fine-tune on each specific speech dataset, restricting their effectiveness in real-world scenes with large-scale noisy data. To address these issues, we propose an active learning (AL) based Fine-Tuning framework for SER that leverages task adaptation pre-training (TAPT) and AL methods to enhance performance and efficiency. Specifically, we first use TAPT to minimize the information gap between the pre-training and the downstream task. Then, AL methods are used to iteratively select a subset of the most informative and diverse samples for fine-tuning, reducing time consumption. Experiments demonstrate that using only 20\%pt. samples improves 8.45\%pt. accuracy and reduces 79\%pt. time consumption.
Cite
@article{arxiv.2310.00283,
title = {Active Learning Based Fine-Tuning Framework for Speech Emotion Recognition},
author = {Dongyuan Li and Yusong Wang and Kotaro Funakoshi and Manabu Okumura},
journal= {arXiv preprint arXiv:2310.00283},
year = {2023}
}