ViSQA: A benchmark dataset and baseline models for Vietnamese spoken question answering

Area of Science:

• Natural Language Processing
• Speech Recognition
• Machine Learning

Background:

• Spoken Question Answering (SQA) integrates automatic speech recognition (ASR) and language understanding.
• Existing SQA benchmarks are limited for low-resource languages like Vietnamese.
• Vietnamese SQA lacks standardized datasets for research and development.

Purpose of the Study:

• Introduce ViSQA, the first benchmark dataset for Vietnamese Spoken Question Answering.
• Enable systematic evaluation of SQA models under varying automatic speech recognition (ASR) transcription quality.
• Facilitate research on the impact of ASR errors on downstream language understanding tasks.

Main Methods:

• Extended the UIT-ViQuAD corpus using a text-to-speech and ASR pipeline.
• Created over 13,000 question-answer pairs with aligned spoken audio.
• Included clean and noise-degraded audio variants for robust evaluation.
• Conducted experiments with five transformer-based models.

Main Results:

• Automatic speech recognition (ASR) errors substantially degrade SQA model performance (e.g., ViT5 EM: 62.04% to 36.30%).
• Training models on spoken transcriptions enhances robustness against ASR errors (e.g., ViT5 EM: 36.30% to 50.70%).
• Demonstrated significant performance drop due to transcription inaccuracies.

Conclusions:

• ViSQA serves as a rigorous benchmark for Vietnamese Spoken Question Answering systems.
• Highlights the critical impact of ASR quality on SQA performance.
• Provides a foundation for developing more robust SQA models for Vietnamese.