Contrastive Similarity Matching for Supervised Learning.

Area of Science:

• Computational Neuroscience
• Deep Learning
• Computer Vision

Background:

• The ventral visual pathway exhibits hierarchical processing where object representations become increasingly category-specific.
• Deep neural networks (DNNs) also learn hierarchical representations, but biologically plausible learning mechanisms remain an active research area.
• The credit assignment problem in DNNs concerns how to effectively propagate error signals to update network weights.

Purpose of the Study:

• To propose a novel, biologically plausible solution to the credit assignment problem in deep neural networks.
• To leverage observations from the ventral visual pathway regarding representational similarity for network learning.
• To develop a learning objective that encourages layer-specific representational similarity interpolation.

Main Methods:

• Formulated a layer-specific learning goal where each layer interpolates representational similarity matrices between adjacent layers.
• Developed a contrastive similarity matching objective function.
• Derived deep neural networks with feedforward, lateral, and feedback connections exhibiting Hebbian and anti-Hebbian plasticity.

Main Results:

• The proposed method results in deep neural networks with biologically plausible neuronal dynamics (Hebbian and anti-Hebbian plasticity).
• Contrastive similarity matching provides an energy-based learning framework with novel contrastive function construction.
• The approach successfully addresses the credit assignment problem by learning hierarchical representations.

Conclusions:

• The proposed contrastive similarity matching offers a biologically plausible mechanism for credit assignment in deep learning.
• This framework bridges computational neuroscience and deep learning by incorporating principles from visual processing.
• The derived network architecture and plasticity rules offer new avenues for developing more efficient and brain-like artificial intelligence.