Using EEG and stimulus context to probe the modelling of auditory-visual speech

Area of Science:

• Neuroscience
• Cognitive Science
• Speech Perception

Background:

• The brain integrates auditory and visual speech cues for robust speech perception.
• The plasticity of these auditory-visual (AV) speech models in response to experience is not fully understood.

Purpose of the Study:

• To investigate whether internal models of AV speech associations can be updated through experience.
• To examine the neural mechanisms underlying AV speech processing and potential experience-dependent changes.

Main Methods:

• Participants experienced valid and invalid AV speech pairings within reliable or unreliable contexts.
• Event-related potentials (ERPs), oscillatory power, and phase locking were measured to index AV associations.
• Neural responses were analyzed in relation to AV validity, context reliability, and visual speech salience.

Main Results:

• N1 facilitation for AV speech occurred in a reliable context but was independent of AV validity.
• Later ERPs and oscillatory power differentiated between valid and invalid AV speech, but these patterns were unaffected by context manipulation.
• N1 facilitation was linked to AV timing rather than visual speech salience.

Conclusions:

• Experience within altered AV speech contexts did not alter neural response patterns, suggesting internal AV speech models are not readily updated.
• The findings highlight the sensitivity of early auditory processing (N1) to AV timing, potentially overriding learned associations.
• This implies a degree of stability in established AV speech representations despite varying sensory input reliability.