AQA-TTRL: Self-Adaptation in Audio Question Answering with Test-Time Reinforcement Learning
Abstract
Large Audio Language Models (LALMs) demonstrate impressive general audio understanding, but once deployed, they are static and fail to improve with new real-world audio data. As traditional supervised fine-tuning is costly, we introduce a novel framework for test-time audio understanding, AQA-TTRL, where an LALM evolves on-the-fly using only unlabeled test data. It first generates pseudo-labels from the prediction via majority voting, then optimizes the model via reinforcement learning. To handle the inherent noise in these self-generated labels, we introduce a confidence-based weighting method to adjust training signals. Furthermore, a multiple-attempt sampling operation mitigates advantage collapse and stabilizes training. On the MMAU (test-mini/test), MMAR, and MMSU benchmarks, AQA-TTRL achieves significant average improvements of 4.42% for the Qwen2.5-Omni 7B model and 11.04% for the 3B model. Notably, the adapted 3B model consistently outperforms the direct inference of the unadapted 7B model, highlighting the effectiveness of previously unexplored test-time adaptations in audio understanding.
Keywords
Cite
@article{arxiv.2510.05478,
title = {AQA-TTRL: Self-Adaptation in Audio Question Answering with Test-Time Reinforcement Learning},
author = {Haoyu Zhang and Jiaxian Guo and Yusuke Iwasawa and Yutaka Matsuo},
journal= {arXiv preprint arXiv:2510.05478},
year = {2026}
}