Crossmodal hierarchical predictive coding for audiovisual sequences in the human brain.

Area of Science:

• Neuroscience
• Cognitive Science
• Computational Neuroscience

Background:

• Predictive coding theory explains sensory processing via brain anticipation using prior knowledge.
• Research on predictive coding is extensive within single sensory systems but limited across modalities.
• Understanding crossmodal predictive processing is crucial for explaining complex cognitive functions.

Purpose of the Study:

• To investigate the neural representation and learning of crossmodal knowledge.
• To identify hierarchical brain networks involved in crossmodal predictions.
• To examine how predictions in one sensory modality influence another.

Main Methods:

• Electroencephalography (EEG) was recorded during a crossmodal audiovisual oddball paradigm.
• Stimulus and sequence-level predictability of audiovisual transitions were manipulated.
• A model-fitting approach was used to analyze neural interactions across modalities and hierarchies.

Main Results:

• Audiovisual integration was observed at both individual stimulus and multi-stimulus sequence levels.
• Spatio-spectro-temporal signatures of crossmodal prediction errors were identified.
• Auditory and visual prediction errors were rapidly redirected to central-parietal areas via alpha-band interactions during learning.

Conclusions:

• The findings support a crossmodal predictive coding mechanism.
• Distributed brain networks process unimodal predictions to build crossmodal knowledge.
• This provides insights into the neural basis of multisensory integration and learning.