English

Leveraging Synthetic Audio Data for End-to-End Low-Resource Speech Translation

Computation and Language 2024-06-28 v2 Sound Audio and Speech Processing
Authors: Yasmin Moslem
View on arXiv ↗ PDF ↗

Abstract

This paper describes our system submission to the International Conference on Spoken Language Translation (IWSLT 2024) for Irish-to-English speech translation. We built end-to-end systems based on Whisper, and employed a number of data augmentation techniques, such as speech back-translation and noise augmentation. We investigate the effect of using synthetic audio data and discuss several methods for enriching signal diversity.

Keywords

speech translation low-resource languages synthetic data Whisper data augmentation

Cite

@article{arxiv.2406.17363,
  title  = {Leveraging Synthetic Audio Data for End-to-End Low-Resource Speech Translation},
  author = {Yasmin Moslem},
  journal= {arXiv preprint arXiv:2406.17363},
  year   = {2024}
}

Comments

IWSLT 2024

Related papers

View all related →
Computation and Language · Computer Science
Bemba Speech Translation: Exploring a Low-Resource African Language
Muhammad Hazim Al Farouq, Aman Kassahun Wassie, Yasmin Moslem
2025-08-14
Computation and Language · Computer Science
Sample, Translate, Recombine: Leveraging Audio Alignments for Data Augmentation in End-to-end Speech Translation
Tsz Kin Lam, Shigehiko Schamoni, Stefan Riezler
2023-06-12
Computation and Language · Computer Science
Finetuning End-to-End Models for Estonian Conversational Spoken Language Translation
Tiia Sildam, Andra Velve, Tanel Alumäe
2024-07-08
Computation and Language · Computer Science
On Using SpecAugment for End-to-End Speech Translation
Parnia Bahar, Albert Zeyer, Ralf Schlüter, Hermann Ney
2019-11-21
Audio and Speech Processing · Electrical Eng. & Systems
ASR data augmentation in low-resource settings using cross-lingual multi-speaker TTS and cross-lingual voice conversion
Edresson Casanova, Christopher Shulby, Alexander Korolev, Arnaldo Candido Junior +3
2023-05-23
Computation and Language · Computer Science
Bridging the Language Gap: Synthetic Voice Diversity via Latent Mixup for Equitable Speech Recognition
Wesley Bian, Xiaofeng Lin, Guang Cheng
2025-11-26
Computation and Language · Computer Science
Speech Synthesis as Augmentation for Low-Resource ASR
Deblin Bagchi, Shannon Wotherspoon, Zhuolin Jiang, Prasanna Muthukumar
2020-12-25
Audio and Speech Processing · Electrical Eng. & Systems
Using Speech Synthesis to Train End-to-End Spoken Language Understanding Models
Loren Lugosch, Brett Meyer, Derek Nowrouzezahrai, Mirco Ravanelli
2019-10-22
Audio and Speech Processing · Electrical Eng. & Systems
Low-Resource Text-to-Speech Synthesis Using Noise-Augmented Training of ForwardTacotron
Kishor Kayyar Lakshminarayana, Frank Zalkow, Christian Dittmar, Nicola Pia +1
2025-06-03
Audio and Speech Processing · Electrical Eng. & Systems
You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentation
Aleksandr Laptev, Roman Korostik, Aleksey Svischev, Andrei Andrusenko +2
2020-12-21
Computation and Language · Computer Science
Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation
Ye Jia, Melvin Johnson, Wolfgang Macherey, Ron J. Weiss +5
2019-02-12
Audio and Speech Processing · Electrical Eng. & Systems
Can Speaker Augmentation Improve Multi-Speaker End-to-End TTS?
Erica Cooper, Cheng-I Lai, Yusuke Yasuda, Junichi Yamagishi
2020-08-10
Audio and Speech Processing · Electrical Eng. & Systems
Noise Robust TTS for Low Resource Speakers using Pre-trained Model and Speech Enhancement
Dongyang Dai, Li Chen, Yuping Wang, Mu Wang +4
2020-10-23
Audio and Speech Processing · Electrical Eng. & Systems
Improving Whispered Speech Recognition Performance using Pseudo-whispered based Data Augmentation
Zhaofeng Lin, Tanvina Patel, Odette Scharenborg
2024-02-08
Audio and Speech Processing · Electrical Eng. & Systems
Generating Data with Text-to-Speech and Large-Language Models for Conversational Speech Recognition
Samuele Cornell, Jordan Darefsky, Zhiyao Duan, Shinji Watanabe
2024-08-20
Audio and Speech Processing · Electrical Eng. & Systems
Cross-speaker style transfer for text-to-speech using data augmentation
Manuel Sam Ribeiro, Julian Roth, Giulia Comini, Goeric Huybrechts +2
2022-02-11
Computation and Language · Computer Science
Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent Synthesis
Jianqiao Lu, Wenyong Huang, Nianzu Zheng, Xingshan Zeng +2
2023-10-25
Computation and Language · Computer Science
Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation
Alexandre Berard, Olivier Pietquin, Christophe Servan, Laurent Besacier
2016-12-07
Computation and Language · Computer Science
One-To-Many Multilingual End-to-end Speech Translation
Mattia Antonino Di Gangi, Matteo Negri, Marco Turchi
2019-10-09
Audio and Speech Processing · Electrical Eng. & Systems
Low-resource expressive text-to-speech using data augmentation
Goeric Huybrechts, Thomas Merritt, Giulia Comini, Bartek Perz +2
2021-06-03
Computation and Language · Computer Science
Elderly-Contextual Data Augmentation via Speech Synthesis for Elderly ASR
Minsik Lee, Seoi Hong, Chongmin Lee, Sieun Choi +3
2026-04-29
Audio and Speech Processing · Electrical Eng. & Systems
Auditory-Based Data Augmentation for End-to-End Automatic Speech Recognition
Zehai Tu, Jack Deadman, Ning Ma, Jon Barker
2022-04-12
Sound · Computer Science
Review of end-to-end speech synthesis technology based on deep learning
Zhaoxi Mu, Xinyu Yang, Yizhuo Dong
2021-04-21
Audio and Speech Processing · Electrical Eng. & Systems
Speechless: Speech Instruction Training Without Speech for Low Resource Languages
Alan Dao, Dinh Bach Vu, Huy Hoang Ha, Tuan Le Duc Anh +5
2025-08-26
Computation and Language · Computer Science
From Start to Finish: Latency Reduction Strategies for Incremental Speech Synthesis in Simultaneous Speech-to-Speech Translation
Danni Liu, Changhan Wang, Hongyu Gong, Xutai Ma +2
2022-07-18