Limited diversity in standardized benchmarks for evaluating audio representation learning (ARL) methods may hinder systematic comparison of current methods' capabilities. We present ARCH, a comprehensive benchmark for evaluating ARL methods on diverse audio classification domains, covering acoustic events, music, and speech. ARCH comprises 12 datasets, that allow us to thoroughly assess pre-trained SSL models of different sizes. ARCH streamlines benchmarking of ARL techniques through its unified access to a wide range of domains and its ability to readily incorporate new datasets and models. To address the current lack of open-source, pre-trained models for non-speech audio, we also release new pre-trained models that demonstrate strong performance on non-speech datasets. We argue that the presented wide-ranging evaluation provides valuable insights into state-of-the-art ARL methods, and is useful to pinpoint promising research directions.
@article{arxiv.2405.00934,
title = {Benchmarking Representations for Speech, Music, and Acoustic Events},
author = {Moreno La Quatra and Alkis Koudounas and Lorenzo Vaiani and Elena Baralis and Luca Cagliero and Paolo Garza and Sabato Marco Siniscalchi},
journal= {arXiv preprint arXiv:2405.00934},
year = {2024}
}