English

Symphony for Speech-to-Text: Supporting Real-Time Medical Voice Interfaces

Machine Learning 2026-05-22 v2 Artificial Intelligence Computation and Language

Abstract

After decades of use in dictation and, more recently, ambient documentation, speech is emerging as a primary modality for interacting with technology and AI in healthcare. Yet medical speech recognition remains difficult: systems must capture specialized terminology, resolve contextual ambiguity, and render measurements, abbreviations, and clinical shorthand precisely. Existing solutions are typically optimized either for general-purpose transcription or narrow dictation workflows, limiting their reliability in safety-critical settings and their usefulness for broader clinical workflows. We introduce Symphony for Speech-to-Text, a medical-grade speech recognition system for real-time streaming and batch file-based clinical use. Symphony decomposes the transcription process into specialized components for recognition, formatting, and contextual correction to optimize medical term recall while producing clinically structured text in real time and adapting across use cases. Evaluations on public benchmark and medical speech datasets show that Symphony substantially outperforms state-of-the-art systems in clinical settings while matching or exceeding them in general-domain settings, suggesting robust generalization rather than overfitting. We release a clinical benchmark dataset to support reliable validation and further progress in medical speech recognition. Symphony is available through a production-grade API for live dictation, conversational transcription, and batch audio file processing.

Keywords

Cite

@article{arxiv.2605.16545,
  title  = {Symphony for Speech-to-Text: Supporting Real-Time Medical Voice Interfaces},
  author = {Arne Nix and Robert James and Lasse Borgholt and Anna B. Ekner and Lana Krumm and Julius Severin and Dan Engel and Lars Maaløe and Jakob Havtorn},
  journal= {arXiv preprint arXiv:2605.16545},
  year   = {2026}
}

Comments

Updated with a correction and improvement to Symphony's performance in spoken punctuation evaluation (R_punct, P_punct)