Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics
Sound
2023-02-28 v1 Computation and Language
Multimedia
Audio and Speech Processing
Abstract
This paper aims to test whether a multi-modal approach for music emotion recognition (MER) performs better than a uni-modal one on high-level song features and lyrics. We use 11 song features retrieved from the Spotify API, combined lyrics features including sentiment, TF-IDF, and Anew to predict valence and arousal (Russell, 1980) scores on the Deezer Mood Detection Dataset (DMDD) (Delbouys et al., 2018) with 4 different regression models. We find that out of the 11 high-level song features, mainly 5 contribute to the performance, multi-modal features do better than audio alone when predicting valence. We made our code publically available.
Keywords
Cite
@article{arxiv.2302.13321,
title = {Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics},
author = {Tibor Krols and Yana Nikolova and Ninell Oldenburg},
journal= {arXiv preprint arXiv:2302.13321},
year = {2023}
}
Comments
12 pages, incl. 2 pages appendix