Contrastive-Equivariant Self-Supervised Learning Improves Alignment with Primate Visual Area IT

Area of Science:

• Computational neuroscience
• Machine learning
• Primate vision

Background:

• Self-supervised learning (SSL) models match supervised models in predicting neural responses.
• SSL is biologically plausible but may create overly invariant representations.
• Network representations need structured variability for better alignment with visual perception.

Purpose of the Study:

• To develop a novel framework for improving SSL models.
• To create contrastive-equivariant losses that preserve input transformations.
• To enhance models' ability to predict neural responses in the primate visual system.

Main Methods:

• Developed a novel framework to convert invariant SSL losses into contrastive-equivariant versions.
• Encouraged preservation of input transformations without supervised parameter access.
• Tested model performance in predicting neural responses in macaque inferior temporal cortex.

Main Results:

• The proposed contrastive-equivariant method systematically increased model performance.
• Models demonstrated enhanced ability to predict neural responses.
• Structured variability in representations improved alignment with visual perception.

Conclusions:

• Incorporating neural computation features into task-optimization builds better models of visual cortex.
• Contrastive-equivariance offers a promising approach for advancing AI models of vision.
• This work bridges machine learning and neuroscience for improved understanding of visual processing.