Contrastive learning explains the emergence and function of visual category-selective regions

Area of Science:

• Neuroscience
• Computational Neuroscience
• Artificial Intelligence

Background:

• Theories of category selectivity in the ventral visual stream have been debated, with modular and distributed coding frameworks in tension.
• Understanding how the brain represents diverse object categories (faces, bodies, scenes, words) remains a key challenge.

Purpose of the Study:

• To reconcile modular and distributed coding theories of category selectivity.
• To propose and validate a contrastive coding framework for explaining category representation in the human brain.

Main Methods:

• Analyzed category selectivity in biological and artificial neural networks using contrastive self-supervised learning.
• Employed unit lesion studies in artificial neural networks to assess functional roles.
• Predicted neural responses in human visual cortex using identified model units and sparse positive encoding.

Main Results:

• Category-selective tuning for faces, bodies, scenes, and words naturally emerged in models trained with contrastive objectives.
• Lesioning model units resulted in selective recognition deficits, supporting distinct functional roles.
• Identified model units successfully predicted neural activity in corresponding human visual cortex regions.

Conclusions:

• Contrastive coding offers a unifying framework for object category emergence and representation.
• Brain-like functional specialization can arise without explicit category-specific learning pressures.
• This approach highlights the power of learning to untangle rich image content for developing specialized representations.