A Visual Encoding Model Based on Contrastive Self-Supervised Learning for Human Brain Activity along the Ventral...

Area of Science:

• Computational neuroscience
• Machine learning
• Neuroimaging

Background:

• Visual encoding models are crucial for understanding visual stream processing.
• Existing models primarily use supervised learning, limiting their scope.
• Unsupervised learning offers a promising alternative for developing more robust models.

Purpose of the Study:

• To construct a visual encoding model using contrastive self-supervised learning for the ventral visual stream.
• To evaluate the performance of this model against supervised approaches using fMRI data.
• To investigate the hierarchical representation capabilities of the self-supervised model.

Main Methods:

• Utilized a ResNet50 model pre-trained with contrastive self-supervised learning (ResNet50-CSL).
• Extracted features from the ResNet50-CSL model.
• Trained linear regression models for each voxel and calculated prediction accuracy using fMRI data.

Main Results:

• The ResNet50-CSL model demonstrated encoding performance comparable or superior to supervised models in visual cortical areas.
• The model exhibited hierarchical representation of visual stimuli, mirroring human visual cortex processing.
• Contrastive self-supervised learning proved effective in extracting brain-like representations.

Conclusions:

• Contrastive self-supervised learning provides a powerful alternative to supervised methods for visual encoding.
• This approach yields effective computational models for understanding visual information processing.
• The findings highlight the potential of self-supervised learning in neuroscience research.