English

GANterpretations

Sound 2020-11-11 v1 Artificial Intelligence Machine Learning Audio and Speech Processing

Abstract

Since the introduction of Generative Adversarial Networks (GANs) [Goodfellow et al., 2014] there has been a regular stream of both technical advances (e.g., Arjovsky et al. [2017]) and creative uses of these generative models (e.g., [Karras et al., 2019, Zhu et al., 2017, Jin et al., 2017]). In this work we propose an approach for using the power of GANs to automatically generate videos to accompany audio recordings by aligning to spectral properties of the recording. This allows musicians to explore new forms of multi-modal creative expression, where musical performance can induce an AI-generated musical video that is guided by said performance, as well as a medium for creating a visual narrative to follow a storyline (similar to what was proposed by Frosst and Kereliuk [2019]).

Keywords

Cite

@article{arxiv.2011.05158,
  title  = {GANterpretations},
  author = {Pablo Samuel Castro},
  journal= {arXiv preprint arXiv:2011.05158},
  year   = {2020}
}

Comments

In 4th Workshop on Machine Learning for Creativity and Design at NeurIPS 2020, Vancouver, Canada

R2 v1 2026-06-23T20:02:59.481Z