Exploring Real-Time Music-to-Image Systems for Creative Inspiration in Music Creation
Abstract
This paper presents a study on the use of a real-time music-to-image system as a mechanism to support and inspire musicians during their creative process. The system takes MIDI messages from a keyboard as input which are then interpreted and analysed using state-of-the-art generative AI models. Based on the perceived emotion and music structure, the system's interpretation is converted into visual imagery that is presented in real-time to musicians. We conducted a user study in which musicians improvised and composed using the system. Our findings show that most musicians found the generated images were a novel mechanism when playing, evidencing the potential of music-to-image systems to inspire and enhance their creative process.
Keywords
Cite
@article{arxiv.2407.05584,
title = {Exploring Real-Time Music-to-Image Systems for Creative Inspiration in Music Creation},
author = {Meng Yang and Maria Teresa Llano and Jon McCormack},
journal= {arXiv preprint arXiv:2407.05584},
year = {2024}
}
Comments
8 pages, 7 figures. Accepted by 15th International Conference on Computational Creativity, ICCC 24