Cross-modal representations in early visual and auditory cortices revealed by multi-voxel pattern analysis

Area of Science:

• Neuroscience
• Cognitive Science
• Sensory Processing

Background:

• Primary sensory cortices process their specific modalities but also respond to cross-modal information.
• Understanding cross-modal processing is key to decoding stimulus categories and the influence of experience and imagination.

Purpose of the Study:

• To investigate cross-modal information processing in the early visual cortex (EVC) and auditory cortex (AC).
• To determine if cross-modal information aids stimulus categorization and the impact of experience on this process.

Main Methods:

• Functional magnetic resonance imaging (fMRI) was used to collect data while participants viewed images and heard sounds.
• Multi-voxel pattern analysis (MVPA) was employed to decode stimulus categories within regions of interest (ROIs).
• Orthogonal fixation tasks restricted imagination and expectations during stimuli presentation.

Main Results:

• Stimulus categories were successfully decoded in EVC (V2, V3) and AC (A1, A2), but not V1.
• Familiar sounds enhanced classification accuracy in V2 and V3 compared to unfamiliar sounds.
• Cross-classification analysis revealed no significant similarity between activity patterns of different modalities.

Conclusions:

• Cross-modal representation is robust in early sensory cortices, even when top-down influences are restricted.
• Auditory experience impacts cross-modal representation in V2 and V3.
• Dissimilar activity patterns between modalities suggest distinct processing pathways in primary sensory cortices.