Manipulating and measuring variation in deep neural network (DNN) representations of objects.

Area of Science:

• Cognitive science
• Computational neuroscience
• Computer vision

Background:

• Deep neural networks (DNNs) generate rich object representations.
• Understanding individual differences in visual cognition is crucial.
• DNNs offer a computational framework for studying these differences.

Purpose of the Study:

• Quantify individual differences in DNN representations.
• Explore the impact of various training manipulations on DNN representations.
• Establish a baseline for representational variation.

Main Methods:

• Systematically explored representational similarity measures: Representational Similarity Analysis (RSA), Centered Kernel Alignment (CKA), and Projection-Weighted Canonical Correlation Analysis (PWCCA).
• Manipulated DNNs by varying random initial weights, training image order, image frequencies, category frequencies, model size, and architecture (All-CNN-C, VGG, ResNet).
• Established a baseline for representational variation using image-augmentation techniques.

Main Results:

• Variations in model randomization and size did not exceed the baseline.
• Differences in training image frequency and category frequencies caused representational variation exceeding the baseline.
• Training category frequency manipulations showed effects earlier in the network layers.

Conclusions:

• Training data characteristics, particularly category frequencies, significantly influence DNN object representations.
• Findings provide insights into the magnitude of representational variations in DNNs.
• This work serves as a foundation for modeling individual differences in high-level visual cognition using DNNs.