We present a system for generating novel lyrics lines conditioned on music audio. A bimodal neural network model learns to generate lines conditioned on any given short audio clip. The model consists of a spectrogram variational autoencoder (VAE) and a text VAE. Both automatic and human evaluations demonstrate effectiveness of our model in generating lines that have an emotional impact matching a given audio clip. The system is intended to serve as a creativity tool for songwriters.
@article{arxiv.2009.14375,
title = {Generation of lyrics lines conditioned on music audio clips},
author = {Olga Vechtomova and Gaurav Sahu and Dhruv Kumar},
journal= {arXiv preprint arXiv:2009.14375},
year = {2020}
}
Comments
Accepted to First Workshop on NLP for Music and Audio (NLP4MusA) at ISMIR 2020