Reassessing hierarchical correspondences between brain and deep networks through direct interface

Area of Science:

• Neuroscience
• Computer Science
• Computational Neuroscience

Background:

• Deep convolutional neural networks (DCNNs) show functional similarities to the mammalian visual system, supporting hierarchical processing models.
• Previous studies focused on shared variance, not task-specific information, limiting the understanding of brain-model correspondences.

Purpose of the Study:

• To propose and test a stricter criterion for functional correspondence between DCNNs and the mammalian visual cortex.
• To investigate whether brain activity can directly drive DCNN object recognition decisions.

Main Methods:

• Developed a novel approach where brain activity replaced model activity to predict object recognition outcomes.
• Utilized three independent datasets for validation.
• Performed time course analyses to examine information flow.

Main Results:

• All regions of the ventral visual stream showed the strongest correspondence with later layers of the DCNN.
• This indicates that all stages of visual processing contain high-level object category information.
• Time course analyses revealed that recurrent connections transmit object class information from later to earlier visual areas.

Conclusions:

• The ventral visual stream processes increasingly complex object information at all stages.
• Recurrent connections play a crucial role in transmitting high-level visual information within the brain.