English

Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Models

Computation and Language 2026-01-07 v1 Audio and Speech Processing

Abstract

Emotion is a central dimension of spoken communication, yet, we still lack a mechanistic account of how modern large audio-language models (LALMs) encode it internally. We present the first neuron-level interpretability study of emotion-sensitive neurons (ESNs) in LALMs and provide causal evidence that such units exist in Qwen2.5-Omni, Kimi-Audio, and Audio Flamingo 3. Across these three widely used open-source models, we compare frequency-, entropy-, magnitude-, and contrast-based neuron selectors on multiple emotion recognition benchmarks. Using inference-time interventions, we reveal a consistent emotion-specific signature: ablating neurons selected for a given emotion disproportionately degrades recognition of that emotion while largely preserving other classes, whereas gain-based amplification steers predictions toward the target emotion. These effects arise with modest identification data and scale systematically with intervention strength. We further observe that ESNs exhibit non-uniform layer-wise clustering with partial cross-dataset transfer. Taken together, our results offer a causal, neuron-level account of emotion decisions in LALMs and highlight targeted neuron interventions as an actionable handle for controllable affective behaviors.

Keywords

Cite

@article{arxiv.2601.03115,
  title  = {Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Models},
  author = {Xiutian Zhao and Björn Schuller and Berrak Sisman},
  journal= {arXiv preprint arXiv:2601.03115},
  year   = {2026}
}

Comments

16 pages, 6 figures

R2 v1 2026-07-01T08:52:48.508Z