Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models
Abstract
Parallel to the development of advanced deepfake audio generation, audio deepfake detection has also seen significant progress. However, a standardized and comprehensive benchmark is still missing. To address this, we introduce Speech DeepFake (DF) Arena, the first comprehensive benchmark for audio deepfake detection. Speech DF Arena provides a toolkit to uniformly evaluate detection systems, currently across 14 diverse datasets and attack scenarios, standardized evaluation metrics and protocols for reproducibility and transparency. It also includes a leaderboard to compare and rank the systems to help researchers and developers enhance their reliability and robustness. We include 14 evaluation sets, 12 state-of-the-art open-source and 3 proprietary detection systems. Our study presents many systems exhibiting high EER in out-of-domain scenarios, highlighting the need for extensive cross-domain evaluation. The leaderboard is hosted on Huggingface1 and a toolkit for reproducing results across the listed datasets is available on GitHub.
Cite
@article{arxiv.2509.02859,
title = {Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models},
author = {Sandipana Dowerah and Atharva Kulkarni and Ajinkya Kulkarni and Hoan My Tran and Joonas Kalda and Artem Fedorchenko and Benoit Fauve and Damien Lolive and Tanel Alumäe and Matthew Magimai Doss},
journal= {arXiv preprint arXiv:2509.02859},
year = {2025}
}