Auditory Perceptual Learning for Speech Perception Can be Enhanced by Audiovisual Training

Area of Science:

• Auditory perception
• Audiovisual speech processing
• Perceptual learning

Background:

• Audiovisual (AV) speech perception typically improves accuracy and speed.
• The impact of AV training on auditory perceptual learning of degraded speech remains unclear.
• Vocoded speech presents a challenge for auditory perception.

Purpose of the Study:

• To investigate how AV training influences auditory perceptual learning of vocoded speech.
• To compare the effectiveness of AV versus audio-only (AO) training methods.
• To explore the underlying mechanisms of AV training's effects on speech perception.

Main Methods:

• Participants underwent paired-associates (PA) training with vocoded nonsense words and pictures.
• Experiment 1 compared AV training with AO training, followed by AO testing.
• Experiment 3 alternated AV and AO training conditions within participants, training to a performance criterion.

Main Results:

• AV training led to significantly higher accuracy in AO testing compared to AO training.
• AV-trained participants showed greater learning of auditory phonetic features than AO-trained participants.
• In contrast, Experiment 3 found AO training to be more effective than AV training.

Conclusions:

• Early AV speech integration may impede auditory perceptual learning.
• Visual top-down access to auditory features can promote auditory perceptual learning.
• Findings are explained by reverse hierarchy theory and the interplay of multisensory and unisensory resources.