Automatic testing of speech recognition.

Area of Science:

• Audiology
• Speech-Language Pathology
• Computational Linguistics

Background:

• Traditional speech reception tests require manual scoring of oral responses, necessitating constant supervisor presence.
• Automated scoring of typed responses offers efficiency but is hindered by spelling errors misidentified as recognition errors.
• Developing robust autocorrection methods is crucial for reliable automated speech testing.

Purpose of the Study:

• To introduce and evaluate two novel autocorrection algorithms for automatically scoring typed speech reception test responses.
• To address the challenge of spelling errors in automated speech recognition (ASR) data.
• To enable accurate and efficient speech reception testing with open-set materials over the internet.

Main Methods:

• Two autocorrection algorithms were developed: one sentence-based using word scores and another word-based using phoneme scores.
• Algorithms were evaluated using a corpus of typed responses from three distinct Dutch speech materials.
• Performance was assessed by comparing automatic scoring to manual scoring, analyzing differences in speech recognition thresholds.

Main Results:

• The sentence-based autocorrection algorithm demonstrated higher accuracy compared to standard methods for the tested speech materials.
• The word-based autocorrection algorithm outperformed human operators in scoring accuracy.
• Both algorithms proved effective in mitigating the impact of spelling errors on test results.

Conclusions:

• The developed autocorrection algorithms provide a viable solution for accurate automated scoring of typed speech reception tests.
• These methods facilitate reliable online speech reception testing, even with open-set speech materials.
• The findings support the practical implementation of automated speech tests, enhancing accessibility and reducing the need for continuous supervision.