Deep supervised, but not unsupervised, models may explain IT cortical representation.

Area of Science:

• Neuroscience
• Computer Vision
• Computational Neuroscience

Background:

• The inferior temporal (IT) cortex is crucial for visual object recognition in primates.
• Current computational models of object vision do not fully replicate human performance or internal representations.

Purpose of the Study:

• To assess how well various computational models explain the representational geometry of the primate IT cortex.
• To identify key features of computational models that best account for IT representations.

Main Methods:

• Compared representational dissimilarity matrices (RDMs) from 37 computational models with human and monkey IT RDMs.
• Evaluated models including HMAX, VisNet, SIFT, GIST, and deep convolutional neural networks (CNNs).
• Used fMRI in humans and cell recordings in monkeys for IT data, with stimuli not used in model training.

Main Results:

• Better performing models showed greater similarity to IT representations, particularly in categorical clustering and within-category dissimilarities.
• Unsupervised models poorly explained the categorical clustering observed in IT.
• A supervised deep CNN achieved the highest performance and best explained IT data, though not fully.

Conclusions:

• Explaining IT representations necessitates computational features trained via supervised learning.
• Emphasizing behaviorally relevant categorical divisions through supervised training is key for IT representation.
• Combining supervised deep CNN features with specific linear combinations fully explained the IT data.