The Effects of Captioning Errors, Background Noise, and Hearing Loss on Memory for Text-Captioned Speech

Area of Science:

• Cognitive Psychology
• Audiology
• Human-Computer Interaction

Background:

• Assistive text captions can mitigate negative effects of acoustic challenge on speech memory.
• Automated speech recognition (ASR) systems commonly produce text-captioning errors.
• The impact of ASR caption errors on speech memory remains under-investigated.

Purpose of the Study:

• To investigate the effects of text-captioning errors on speech memory.
• To examine the interaction between caption errors, background noise, and hearing acuity on memory.
• To determine the differential impact of caption errors on young and older adults.

Main Methods:

• Two experiments were conducted involving young and older adults.
• Participants recalled text-captioned speech under conditions with and without background noise.
• Captioning conditions included error-free captions and captions with substitution, deletion, or insertion errors.

Main Results:

• Both background noise and captioning errors negatively impacted speech memory.
• The detrimental effects of caption errors and noise were additive.
• Older adults with greater hearing loss experienced the most significant negative effects from caption errors.

Conclusions:

• Even single-word caption errors can impair memory for text-captioned speech.
• The negative impact of caption errors is amplified in older adults with hearing loss.
• Accurate text captions are essential for maximizing memory benefits, particularly for hearing-impaired individuals.