Individual differences among deep neural network models.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Systems Neuroscience

Background:

• Deep neural networks (DNNs) are increasingly utilized to model primate brain computations.
• Individual DNNs, like brains, exhibit unique connectivity and representational profiles.

Purpose of the Study:

• To investigate how random weight initialization affects DNN representations.
• To determine if DNNs trained with different initializations yield comparable insights into neural processing.

Main Methods:

• Trained multiple DNN instances with identical architectures but varied random weight initializations.
• Analyzed intermediate and higher-level network representations using systems neuroscience tools.
• Examined the alignment of category exemplars and centroids within DNNs.

Main Results:

• Substantial differences in network representations were observed across DNN instances.
• These representational differences persisted despite similar overall classification performance.
• The variations originated from under-constrained alignment of category exemplars, not misaligned centroids.

Conclusions:

• Single DNN instances may not reliably represent neural information processing.
• Computational neuroscientists should consider using ensembles of multiple DNNs for robust inferences.
• Random initialization is a critical factor influencing DNN representational geometry.