Seeing it all: Convolutional network layers map the function of the human visual system.

Area of Science:

• Neuroscience
• Computer Vision
• Artificial Intelligence

Background:

• Convolutional neural networks (CNNs) are computational models inspired by the mammalian visual system.
• Understanding the neural computations in the human brain during visual processing is a key challenge.

Purpose of the Study:

• To model human brain activity during natural image viewing using CNNs.
• To investigate the relationship between CNN layers and brain activity in visual cortical areas.

Main Methods:

• Constructing predictive models of brain activity based on CNN layers and BOLD fMRI data.
• Analyzing the predictive performance of different CNN layers across visual brain regions.
• Synthesizing brain activity using a deep encoding model for validation.

Main Results:

• Characteristic 'fingerprints' for visual brain regions were identified based on CNN layer performance.
• A hierarchical progression was observed, with early visual areas corresponding to lower CNN layers and later areas to higher layers.
• The model successfully generalized to different experimental paradigms, including retinotopy and face-place stimuli.

Conclusions:

• CNNs provide a powerful framework for modeling visual processing in the human brain.
• The hierarchical organization of CNNs mirrors the functional organization of the visual cortex.
• This deep encoding model captures universal representations of brain function across diverse visual tasks.