English

TAC: Timestamped Audio Captioning

Sound 2026-02-18 v1

Abstract

Large Audio Language Models struggle to disentangle overlapping events in complex acoustic scenes, yielding temporally inconsistent captions and frequent hallucinations. We introduce Timestamped Audio Captioner (TAC), a model that produces temporally grounded audio descriptions at varying degrees of detail and resolution. TAC is trained with a synthetic data pipeline that constructs challenging and dynamic mixtures from real-world audio sources, enabling robust learning under realistic polyphonic conditions. Across event detection and dense captioning, TAC outperforms all competing methods, with a low hallucination rate and accurate temporal grounding. We also introduce TAC-V, an audio-visual pipeline to generate semantically rich audio-visual descriptions. We then show that TAC and TAC-V serves as a "semantic bridge" for a text-only reasoner: a simple TAC\rightarrowLLM and TAC-V\rightarrowLLM cascade achieves state-of-the-art scores on benchmarks for both audio (MMAU-Pro, MMSU, MMAR) and audio-visual (DailyOmni, VideoHolmes) understanding and reasoning respectively.

Keywords

Cite

@article{arxiv.2602.15766,
  title  = {TAC: Timestamped Audio Captioning},
  author = {Sonal Kumar and Prem Seetharaman and Ke Chen and Oriol Nieto and Jiaqi Su and Zhepei Wang and Rithesh Kumar and Dinesh Manocha and Nicholas J. Bryan and Zeyu Jin and Justin Salamon},
  journal= {arXiv preprint arXiv:2602.15766},
  year   = {2026}
}
R2 v1 2026-07-01T10:40:13.784Z