Prior multisensory learning can facilitate auditory-only voice-identity and speech recognition in noise

Area of Science:

• Auditory perception
• Neuroscience
• Speech processing

Background:

• Multisensory enhancement improves speech understanding, particularly in noise.
• The "face-benefit" shows auditory-only speech and voice recognition improve when speakers are learned with their face.
• It is unknown if this face-benefit aids perception in noisy conditions.

Purpose of the Study:

• To investigate if the face-benefit helps maintain robust auditory perception in increasing noise.
• To examine the effect of learned facial information on speech and voice-identity recognition under varying noise levels.

Main Methods:

• Two behavioral experiments were conducted.
• Participants learned speakers' voices with dynamic faces or control images.
• Auditory-only speech and voice-identity recognition were tested in increasing auditory noise.

Main Results:

• A face-benefit was observed in 47% of participants for speech recognition and 76% for voice-identity recognition.
• For participants showing a face-benefit, its magnitude increased with higher auditory noise levels.
• Results suggest learned facial characteristics help manage auditory uncertainty.

Conclusions:

• The findings support an audio-visual model of auditory communication.
• The brain can flexibly use learned facial features to compensate for auditory uncertainty.
• This research highlights the role of visual learning in auditory processing robustness.