Neural correlates of audiovisual speech processing in a second language

Area of Science:

• Neuroscience
• Cognitive Science
• Speech Perception

Background:

• Previous research on audiovisual (AV) speech processing focused solely on native language (L1) perception.
• Behavioral studies indicate AV processing is also crucial for non-native (L2) speech perception.

Purpose of the Study:

• To investigate brain activity differences during L1 and L2 audiovisual speech processing using fMRI.
• To determine if audiovisual integration mechanisms differ based on language background.

Main Methods:

• Functional magnetic resonance imaging (fMRI) was used to measure brain activity.
• Participants listened to auditory, visual, and both congruent and incongruent audiovisual speech stimuli in their L1 and L2.
• Blood-oxygen-level-dependent (BOLD) responses were analyzed.

Main Results:

• Congruent AV speech elicited stronger BOLD responses in the posterior superior temporal sulcus (pSTS) than unimodal stimuli, irrespective of language (L1 or L2).
• No significant differences in AV processing within the pSTS were observed between L1 and L2.
• The bilateral occipital lobe showed a greater congruency effect (congruent > incongruent) for L2 compared to L1 speech.

Conclusions:

• The pSTS is consistently involved in audiovisual speech integration, regardless of the listener's language background.
• Language background modulates audiovisual processing in unimodal cortical regions, specifically the bilateral occipital lobe.
• These findings highlight distinct neural mechanisms for audiovisual speech processing in native versus non-native languages.