Deep neural networks rival the representation of primate IT cortex for core visual object recognition

Area of Science:

• Neuroscience
• Computer Science
• Artificial Intelligence

Background:

• The primate visual system excels at core visual object recognition, a feat mediated by the inferior temporal (IT) cortex.
• Deep neural networks (DNNs) have advanced object recognition, but their representational performance compared to the brain remains unclear.

Purpose of the Study:

• To directly compare the representational performance of DNNs with primate IT cortex for visual object recognition.
• To address limitations in previous comparison metrics, including experimental and computational factors.

Main Methods:

• Developed an extension of kernel analysis to measure generalization accuracy against representational complexity.
• Performed a direct comparison accounting for noise, recording sites, trials, classifier complexity, and training examples.

Main Results:

• The latest DNNs demonstrate representational performance rivaling that of primate IT cortex.
• DNNs with high representational performance also show high similarity to IT representations and predict IT neural responses.

Conclusions:

• Unlike prior bio-inspired models, current DNNs' representational capabilities cannot be dismissed as dissimilar to the primate visual system.
• The possibility of similar computational mechanisms underlying DNN and primate visual object recognition remains open.