Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Models
Abstract
Emotion is a central dimension of spoken communication, yet, we still lack a mechanistic account of how modern large audio-language models (LALMs) encode it internally. We present the first neuron-level interpretability study of emotion-sensitive neurons (ESNs) in LALMs and provide causal evidence that such units exist in Qwen2.5-Omni, Kimi-Audio, and Audio Flamingo 3. Across these three widely used open-source models, we compare frequency-, entropy-, magnitude-, and contrast-based neuron selectors on multiple emotion recognition benchmarks. Using inference-time interventions, we reveal a consistent emotion-specific signature: ablating neurons selected for a given emotion disproportionately degrades recognition of that emotion while largely preserving other classes, whereas gain-based amplification steers predictions toward the target emotion. These effects arise with modest identification data and scale systematically with intervention strength. We further observe that ESNs exhibit non-uniform layer-wise clustering with partial cross-dataset transfer. Taken together, our results offer a causal, neuron-level account of emotion decisions in LALMs and highlight targeted neuron interventions as an actionable handle for controllable affective behaviors.
Cite
@article{arxiv.2601.03115,
title = {Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Models},
author = {Xiutian Zhao and Björn Schuller and Berrak Sisman},
journal= {arXiv preprint arXiv:2601.03115},
year = {2026}
}
Comments
16 pages, 6 figures