English

Multilingual Simultaneous Speech Translation

Computation and Language 2022-03-30 v2 Sound Audio and Speech Processing

Abstract

Applications designed for simultaneous speech translation during events such as conferences or meetings need to balance quality and lag while displaying translated text to deliver a good user experience. One common approach to building online spoken language translation systems is by leveraging models built for offline speech translation. Based on a technique to adapt end-to-end monolingual models, we investigate multilingual models and different architectures (end-to-end and cascade) on the ability to perform online speech translation. On the multilingual TEDx corpus, we show that the approach generalizes to different architectures. We see similar gains in latency reduction (40% relative) across languages and architectures. However, the end-to-end architecture leads to smaller translation quality losses after adapting to the online model. Furthermore, the approach even scales to zero-shot directions.

Keywords

Cite

@article{arxiv.2203.14835,
  title  = {Multilingual Simultaneous Speech Translation},
  author = {Shashank Subramanya and Jan Niehues},
  journal= {arXiv preprint arXiv:2203.14835},
  year   = {2022}
}

Comments

Submitted to Interspeech 2022

R2 v1 2026-06-24T10:28:32.942Z