Closing the gap between single-unit and neural population codes: Insights from deep learning in face recognition.

Area of Science:

• Computational neuroscience
• Computer vision
• High-level visual processing

Background:

• Single-unit and population codes offer distinct information about visual representations.
• Reconciling local and global read-outs with in vivo methods is challenging.
• Deep convolutional neural networks (DCNNs) trained for face recognition offer a model system.

Purpose of the Study:

• To investigate the relationship between single-unit and ensemble codes for facial identity, gender, and viewpoint.
• To compare representations in DCNNs with primate visual systems.
• To assess the predictive power of individual units and network ensembles for facial attributes.

Main Methods:

• Utilized a DCNN trained for face recognition.
• Analyzed single-unit responses to determine attribute predictability.
• Employed principal component analysis (PCA) on network representations.
• Compared unit-based directions with attribute-associated directions.

Main Results:

• Accurate face identification was achieved with only 3% of output units; individual units showed substantial identity-predicting power.
• Cross-unit responses were minimally correlated, suggesting non-redundant identity coding.
• Gender and viewpoint classification required large-scale unit pooling, with weak individual unit predictive power.
• Identity, gender, and viewpoint contributed to all unit responses, indicating superimposed, distributed codes.

Conclusions:

• Single-unit responses in DCNNs represent superimposed, distributed codes for facial identity, gender, and viewpoint.
• This challenges the neural tuning analogy for high-level visual representations.
• Findings question the interpretation of neural representations from unit response profiles in both DCNNs and biological vision.