Causal importance of low-level feature selectivity for generalization in image recognition

Area of Science:

• Neuroscience and Machine Learning
• Computational Vision
• Deep Learning Architectures

Background:

• Deep neural networks (DNNs) and brains process information hierarchically, but mechanisms remain unclear.
• Previous work suggested class-selective units in DNNs may hinder generalization.
• Understanding unit selectivity is key to improving AI and brain models.

Purpose of the Study:

• Revisit the role of unit selectivity in DNN generalization.
• Investigate the impact of low-level feature selectivity, specifically orientation selectivity.
• Determine if selective units are always detrimental to network performance.

Main Methods:

• Analyzed orientation selectivity of units in DNNs trained for image classification.
• Examined unit selectivity across different network layers during training.
• Assessed the effect of ablating selective units on network generalization performance.

Main Results:

• Orientation-selective units were found in both lower and higher layers of DNNs.
• Lower-layer units showed increased orientation selectivity correlating with improved generalization.
• Networks with better generalization exhibited higher orientation selectivity in lower layers.
• Ablating lower-layer selective units significantly impaired generalization by disrupting higher-layer shift-invariance.

Conclusions:

• Contrary to prior hypotheses, low-level selective units are vital for DNN generalization.
• Orientation selectivity in lower layers plays a causal role in object recognition.
• These findings highlight the indispensable nature of low-level feature selectivity for generalization in DNNs.