Convergent and invariant object representations for sight, sound, and touch

Area of Science:

• Neuroscience
• Cognitive Neuroscience
• Sensory Integration

Background:

• Humans perceive the world through multiple sensory channels simultaneously.
• Understanding how the brain integrates this multisensory information is crucial for comprehending perception and cognition.

Purpose of the Study:

• To investigate the convergence of information from visual, auditory, and tactile sensory streams within the cerebral cortex.
• To identify brain regions involved in representing object identity across different sensory modalities.

Main Methods:

• Multivariate pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data.
• Presented volunteers with common objects via sight, sound, and touch.
• Performed intramodal and crossmodal classification analyses.

Main Results:

• Neural activity patterns in early sensory cortices predicted object identity for seen, heard, or touched stimuli.
• Multisensory convergence was observed in association cortices beyond primary sensory areas.
• Crossmodal generalization revealed audiovisual invariance in the temporo-occipital junction, audiotactile invariance in the postcentral gyrus/parietal operculum, and visuotactile invariance in the postcentral/supramarginal gyri.

Conclusions:

• The study maps multisensory convergence and crossmodal generalization in the human brain.
• Findings illuminate the organization of association cortices and their role in forming mental concepts.
• Identified specific brain regions supporting cross-sensory object representation.