Brain-optimized deep neural network models of human visual areas learn non-hierarchical representations

Area of Science:

• Computational neuroscience
• Cognitive neuroscience
• Artificial intelligence

Background:

• Deep neural networks (DNNs) trained for visual tasks develop hierarchical representations mirroring primate visual cortex organization.
• A prevailing interpretation suggests these hierarchical representations are essential for accurately modeling primate visual system activity.

Purpose of the Study:

• To investigate whether hierarchical representations are a necessary component for accurately predicting human brain activity in early visual areas (V1-V4).
• To compare the predictive power of single-branch versus multi-branch DNN architectures on fMRI data from human visual cortex.

Main Methods:

• Trained deep neural networks (DNNs) to directly predict brain activity measured via functional magnetic resonance imaging (fMRI) in human visual areas V1-V4.
• Employed two distinct DNN architectures: a single-branch network predicting all areas jointly and a multi-branch network predicting each area independently.

Main Results:

• The multi-branch DNN, predicting visual areas independently, demonstrated the capacity to learn hierarchical representations.
• However, only the single-branch DNN, predicting all visual areas jointly, inherently developed hierarchical representations.
• Both architectures achieved accurate predictions of human brain activity in V1-V4, irrespective of explicit hierarchical structure.

Conclusions:

• Hierarchical representations are not strictly necessary for deep neural networks (DNNs) to accurately predict human brain activity in visual areas V1-V4.
• DNNs encoding brain-like visual representations can exhibit diverse architectures, including both serial hierarchies and independent processing branches.