Generative Modelling for Controllable Audio Synthesis of Expressive Piano Performance
Audio and Speech Processing
2020-07-14 v2 Machine Learning
Multimedia
Sound
Abstract
We present a controllable neural audio synthesizer based on Gaussian Mixture Variational Autoencoders (GM-VAE), which can generate realistic piano performances in the audio domain that closely follows temporal conditions of two essential style features for piano performances: articulation and dynamics. We demonstrate how the model is able to apply fine-grained style morphing over the course of synthesizing the audio. This is based on conditions which are latent variables that can be sampled from the prior or inferred from other pieces. One of the envisioned use cases is to inspire creative and brand new interpretations for existing pieces of piano music.
Cite
@article{arxiv.2006.09833,
title = {Generative Modelling for Controllable Audio Synthesis of Expressive Piano Performance},
author = {Hao Hao Tan and Yin-Jyun Luo and Dorien Herremans},
journal= {arXiv preprint arXiv:2006.09833},
year = {2020}
}