Neural Correlates of Modality-Sensitive Deviance Detection in the Audiovisual Oddball Paradigm

Area of Science:

• Neuroscience
• Psychology
• Audiology

Background:

• The McGurk effect demonstrates audiovisual speech perception, where incongruent visual and auditory stimuli create a fused percept.
• Previous research links audiovisual fusion illusions to mismatch negativity (MMN) in auditory cortex, even without acoustic changes.

Purpose of the Study:

• To investigate how visual and auditory components of incongruent speech stimuli influence the audiovisual mismatch negativity (MMN).
• To compare the neural responses to fusion and combination illusions in a passive oddball paradigm.

Main Methods:

• A passive oddball paradigm was used with audiovisual stimuli under fusion and combination illusion conditions.
• Participants viewed stimuli where either the visual or auditory aspect changed, alongside unimodal auditory and visual conditions.
• Event-related potentials (ERPs), specifically MMN, were recorded to analyze neural responses.

Main Results:

• Both fusion and combination illusions generated congruency predictions, with significant differences between standards and deviants in the N100 time window.
• The MMN response differed between fusion and combination illusions across early and late time windows.
• A changing visual signal created a new percept, whereas a changing auditory signal with a constant visual input suppressed the response, eliciting a later MMN.

Conclusions:

• Visual information plays a critical role in predicting auditory speech perception.
• Visual input can suppress auditory speech perception, supporting predictive processing models.
• The timing and nature of stimulus changes (visual vs. auditory) differentially impact audiovisual MMN responses.