Teaching deep networks to see shape: Lessons from a simplified visual world

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Computer vision

Background:

• Deep neural networks (DNNs) are successful models of the primate visual system.
• DNNs exhibit a strong shape-dependence in human vision, unlike current models.
• Humans prioritize shape for category judgments, while DNNs favor color and texture.

Purpose of the Study:

• Investigate why DNNs fail to capture the shape-dependence of primate vision.
• Identify the underlying reasons for DNNs' bias towards non-shape features.
• Propose solutions to enhance DNNs' sensitivity to shape.

Main Methods:

• Designed artificial image datasets with isolated shape, color, and texture features.
• Trained DNNs from scratch on these datasets using single features and combinations.
• Analyzed network architectures and learning algorithms, specifically mini-batch gradient descent.

Main Results:

• Some DNN architectures were unable to learn shape features effectively.
• Other architectures showed a bias towards color and texture, despite being capable of learning shape.
• This bias was linked to weight update interactions during mini-batch gradient descent.

Conclusions:

• Current DNNs and learning algorithms are not optimized for shape-based visual processing.
• Mini-batch gradient descent contributes to the bias against shape features.
• Developing learning algorithms with sparser, more local weight changes is crucial for improving DNNs' shape sensitivity and modeling human vision.