Improving data augmentation for low resource speech-to-text translation with diverse paraphrasing

Area of Science:

• Natural Language Processing
• Speech Technology
• Machine Translation

Background:

• High-quality speech translation models require extensive speech-to-text data, which is often unavailable for low-resource languages.
• Existing methods struggle to address data scarcity in low-resource speech translation scenarios.

Purpose of the Study:

• To develop a target-side data augmentation method to enhance low-resource speech translation.
• To improve the performance of end-to-end speech translation models for languages with limited training data.

Main Methods:

• Generated large-scale target-side paraphrases using a model combining statistical machine translation (SMT) and recurrent neural network (RNN) features.
• Employed a filtering model based on semantic similarity and speech-word co-occurrence to select high-quality source speech-target paraphrase pairs.
• Implemented two strategies—audio-text pair recombination and multiple references training—to integrate paraphrase generation results into speech translation.

Main Results:

• The proposed paraphrase generation method demonstrated significant and consistent improvements on PPDB datasets across multiple languages.
• Speech translation models trained with the augmented data showed substantial performance gains, particularly for low-resource language pairs.
• The data augmentation approach effectively addresses the challenge of data scarcity in speech translation.

Conclusions:

• The proposed target-side data augmentation method is effective for improving low-resource speech translation.
• Combining paraphrase generation with specific training strategies offers a viable solution for data-scarce language pairs.
• This work contributes to advancing speech translation capabilities for a wider range of languages.