The Cross-Modal Suppressive Role of Visual Context on Speech Intelligibility: An ERP Study

Area of Science:

• Neuroscience
• Cognitive Science
• Psychology

Background:

• Audiovisual (AV) integration relies on cross-modal interactions, affecting auditory event-related potentials (ERPs).
• Semantic encoding strength correlates with late ERP negativities, like N450.
• Visual stimulus reliability is hypothesized to influence AV integration and semantic prediction.

Purpose of the Study:

• To investigate how visual stimulus reliability affects AV integration and semantic prediction.
• To examine the impact of visual clarity on auditory ERPs and late negativities.

Main Methods:

• Electroencephalography (EEG) recorded while participants viewed clear/blurred videos of intact/vocoded speech.
• Participants made semantic judgments (animate/inanimate) on spoken words.
• Auditory ERPs (P1-N1-P2) and late negativities (N450) analyzed.

Main Results:

• Intact speech elicited greater late negativity than vocoded speech, indicating stronger semantic prediction.
• Clearer videos led to greater cross-modal ERP suppression before sound onset and for P2 ERP.
• Late semantic-related negativity was larger for clear videos, though tempered by vocoded speech.

Conclusions:

• Cross-modal effects are mainly in early auditory network suppression, with limited impact on semantic prediction networks.
• Visual clarity enhances AV integration but has a weaker influence on semantic processing, especially with degraded speech signals.