Hierarchical Sparse Coding of Objects in Deep Convolutional Neural Networks.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Computer Vision

Background:

• Deep convolutional neural networks (DCNNs) achieve high performance in object recognition but their internal representations are not fully understood.
• The primate brain represents objects using distributed, local, or sparse coding schemes.
• Understanding DCNN object representation could reveal principles shared with biological systems.

Purpose of the Study:

• To investigate whether DCNNs employ sparse coding for object representation.
• To analyze how object representation changes across DCNN layers.
• To correlate coding schemes with DCNN performance and identify factors influencing representation development.

Main Methods:

• Applied the population sparseness index to analyze DCNN layers.
• Examined DCNNs pretrained for object categorization.
• Utilized a lesion approach to assess the impact of learning and internal operations.

Main Results:

• DCNNs consistently use sparse coding across all layers.
• Sparseness increases with network depth, shifting from distributed-like to local-like coding.
• Higher sparseness correlates positively with improved object categorization performance.
• Both external learning and internal gating mechanisms are crucial for developing hierarchical sparse coding.

Conclusions:

• DCNNs adopt a hierarchical sparse coding scheme analogous to the biological brain.
• This suggests an implementation-independent principle underlying object recognition in both artificial and biological systems.
• The findings open avenues for understanding neural computation and developing more efficient AI.