Identifying Hearing Difficulty Moments in Conversational Audio
Abstract
Individuals regularly experience Hearing Difficulty Moments in everyday conversation. Identifying these moments of hearing difficulty has particular significance in the field of hearing assistive technology where timely interventions are key for realtime hearing assistance. In this paper, we propose and compare machine learning solutions for continuously detecting utterances that identify these specific moments in conversational audio. We show that audio language models, through their multimodal reasoning capabilities, excel at this task, significantly outperforming a simple ASR hotword heuristic and a more conventional fine-tuning approach with Wav2Vec, an audio-only input architecture that is state-of-the-art for automatic speech recognition (ASR).
Cite
@article{arxiv.2507.23590,
title = {Identifying Hearing Difficulty Moments in Conversational Audio},
author = {Jack Collins and Adrian Buzea and Chris Collier and Alejandro Ballesta Rosen and Julian Maclaren and Richard F. Lyon and Simon Carlile},
journal= {arXiv preprint arXiv:2507.23590},
year = {2025}
}