Audio diffusion models can synthesize high-fidelity music from text, yet achieving fine-grained control over specific musical attributes remains challenging, as their internal mechanisms for representing high-level concepts are poorly understood. In this work, we use activation patching to demonstrate that recent audio diffusion architectures exhibit a semantic bottleneck, where a small, shared subset of consecutive attention layers controls distinct musical concepts, such as the presence of specific instruments, vocals, or genres. Building on this, we systematically evaluate a broad spectrum of steering paradigms, comparing activation steering against prompt-level, score-space, and weight-space interventions, analyzing the interaction between the steering mechanism and the intervention site. Our new benchmark, supported by an extensive user study, demonstrates that localized activation steering establishes a new state-of-the-art in audio concept modulation.
@article{arxiv.2602.11910,
title = {TADA! Tuning Audio Diffusion Models through Activation Steering},
author = {Łukasz Staniszewski and Katarzyna Zaleska and Mateusz Modrzejewski and Kamil Deja},
journal= {arXiv preprint arXiv:2602.11910},
year = {2026}
}