Audiovisual perceptual learning with multiple speakers.

Area of Science:

• Psycholinguistics
• Auditory Perception
• Speech Processing

Background:

• Speech perception faces challenges due to acoustic variability between different speakers.
• Listeners must interpret acoustically distinct speech sounds as equivalent phonemes.
• Perceptual tuning, using context to adjust phonemic representations, aids in overcoming speaker variability.

Purpose of the Study:

• To investigate if visual cues of speaker identity facilitate speaker-specific perceptual tuning.
• To determine if facial identity information helps listeners form and maintain distributional representations for individual speakers.
• To examine how facial cues influence the adjustment of phoneme boundaries in a speaker-specific manner.

Main Methods:

• Participants were familiarized with an audiovisual speech continuum between /aba/ and /ada/.
• Distinct faces ('b-face' and 'D-face') were presented mouthing /aba/ and /ada/ respectively, paired with ambiguous speech tokens.
• Listener responses to ambiguous tokens were tested when paired with static images of the familiarized faces.

Main Results:

• Ambiguous speech tokens were more frequently identified as /aba/ when paired with the 'b-face' image compared to the 'D-face' image.
• This effect was absent in a control condition where faces were equally paired with ambiguous tokens.
• Results indicate a speaker-specific adjustment in phoneme boundary perception.

Conclusions:

• Listeners can form speaker-specific phonemic representations by utilizing visual facial identity cues.
• Facial cues play a role in enabling perceptual tuning, allowing listeners to adapt to speaker variability.
• This suggests a multimodal integration process in speech perception, where visual information modulates auditory processing.