English

Audio Latent Space Cartography

Sound 2022-12-08 v2 Machine Learning Audio and Speech Processing
Authors: Nicolas Jonason , Bob L. T. Sturm
View on arXiv ↗ PDF ↗

Abstract

We explore the generation of visualisations of audio latent spaces using an audio-to-image generation pipeline. We believe this can help with the interpretability of audio latent spaces. We demonstrate a variety of results on the NSynth dataset. A web demo is available.

Keywords

audio generation music information retrieval

Cite

@article{arxiv.2212.02610,
  title  = {Audio Latent Space Cartography},
  author = {Nicolas Jonason and Bob L. T. Sturm},
  journal= {arXiv preprint arXiv:2212.02610},
  year   = {2022}
}

Comments

Late Breaking / Demo, ISMIR 2022 (https://ismir2022program.ismir.net/lbd_413.html)

Related papers

View all related →
Computer Vision and Pattern Recognition · Computer Science
Sound to Visual Scene Generation by Audio-to-Visual Latent Alignment
Kim Sung-Bin, Arda Senocak, Hyunwoo Ha, Andrew Owens +1
2023-03-31
Computer Vision and Pattern Recognition · Computer Science
Sound2Vision: Generating Diverse Visuals from Audio through Cross-Modal Latent Alignment
Kim Sung-Bin, Arda Senocak, Hyunwoo Ha, Tae-Hyun Oh
2024-12-10
Computer Vision and Pattern Recognition · Computer Science
Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners
Yazhou Xing, Yingqing He, Zeyue Tian, Xintao Wang +1
2024-02-29
Sound · Computer Science
Seeing Sound: Assembling Sounds from Visuals for Audio-to-Image Generation
Darius Petermann, Mahdi M. Kalayeh
2025-01-10
Sound · Computer Science
Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation
Peiwen Sun, Sitong Cheng, Xiangtai Li, Zhen Ye +4
2025-02-26
Sound · Computer Science
From Sound to Sight: Towards AI-authored Music Videos
Leo Vitasovic, Stella Graßhof, Agnes Mercedes Kloft, Ville V. Lehtola +5
2025-09-03
Sound · Computer Science
STASE: A spatialized text-to-audio synthesis engine for music generation
Tutti Chi, Letian Gao, Yixiao Zhang
2025-09-16
Sound · Computer Science
A Mapping Strategy for Interacting with Latent Audio Synthesis Using Artistic Materials
Shuoyang Zheng, Anna Xambó Sedó, Nick Bryan-Kinns
2024-07-22
Computer Vision and Pattern Recognition · Computer Science
SonicDiffusion: Audio-Driven Image Generation and Editing with Pretrained Diffusion Models
Burak Can Biner, Farrin Marouf Sofian, Umur Berkay Karakaş, Duygu Ceylan +2
2024-05-03
Machine Learning · Computer Science
Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders
Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman +3
2017-04-06
Sound · Computer Science
What Do I Hear? Generating Sounds for Visuals with ChatGPT
David Chuan-En Lin, Nikolas Martelaro
2023-11-10
Computer Vision and Pattern Recognition · Computer Science
SpatialV2A: Visual-Guided High-fidelity Spatial Audio Generation
Yanan Wang, Linjie Ren, Zihao Li, Junyi Wang +1
2026-01-30
Computer Vision and Pattern Recognition · Computer Science
Narrating For You: Prompt-guided Audio-visual Narrating Face Generation Employing Multi-entangled Latent Space
Aashish Chandra, Aashutosh A, Abhijit Das
2026-02-24
Sound · Computer Science
A Framework for Synthetic Audio Conversations Generation using Large Language Models
Kaung Myat Kyaw, Jonathan Hoyin Chan
2025-07-08
Audio and Speech Processing · Electrical Eng. & Systems
SemanticAudio: Audio Generation and Editing in Semantic Space
Zheqi Dai, Guangyan Zhang, Haolin He, Xiquan Li +4
2026-01-30
Sound · Computer Science
Challenge on Sound Scene Synthesis: Evaluating Text-to-Audio Generation
Junwon Lee, Modan Tailleur, Laurie M. Heller, Keunwoo Choi +4
2024-10-24
Multimedia · Computer Science
Visual-based spatial audio generation system for multi-speaker environments
Xiaojing Liu, Ogulcan Gurelli, Yan Wang, Joshua Reiss
2025-02-14
Computer Vision and Pattern Recognition · Computer Science
Sound-Guided Semantic Video Generation
Seung Hyun Lee, Gyeongrok Oh, Wonmin Byeon, Chanyoung Kim +6
2022-10-24
Computation and Language · Computer Science
Improving Text-To-Audio Models with Synthetic Captions
Zhifeng Kong, Sang-gil Lee, Deepanway Ghosal, Navonil Majumder +4
2024-07-10
Graphics · Computer Science
SpA2V: Harnessing Spatial Auditory Cues for Audio-driven Spatially-aware Video Generation
Kien T. Pham, Yingqing He, Yazhou Xing, Qifeng Chen +1
2026-03-17
Computer Vision and Pattern Recognition · Computer Science
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation
Hang Zhou, Yu Liu, Ziwei Liu, Ping Luo +1
2019-04-24
Sound · Computer Science
ArchiSound: Audio Generation with Diffusion
Flavio Schneider
2023-02-01
Sound · Computer Science
Timbre latent space: exploration and creative aspects
Antoine Caillon, Adrien Bitton, Brice Gatinet, Philippe Esling
2020-08-18
Sound · Computer Science
Text2Move: Text-to-moving sound generation via trajectory prediction and temporal alignment
Yunyi Liu, Shaofan Yang, Kai Li, Xu Li
2025-09-29
Computation and Language · Computer Science
Towards Audio to Scene Image Synthesis using Generative Adversarial Network
Chia-Hung Wan, Shun-Po Chuang, Hung-Yi Lee
2018-08-14
R2 v1 2026-06-28T07:22:58.566Z