Learning Credit Assignment.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Deep learning models demonstrate high prediction accuracy across various domains.
• The complex, nonlinear nature of deep learning hinders understanding of its internal decision-making processes and parameter coordination.

Purpose of the Study:

• To elucidate the hierarchical credit assignment in deep learning.
• To develop a model that explains how deep learning coordinates numerous parameters for classification tasks.

Main Methods:

• Proposed a mean-field learning model for classification tasks.
• Assumed an ensemble of subnetworks is trained instead of a single network.
• Analyzed the distribution and roles of synaptic weights between neurons in neighboring layers.

Main Results:

• Identified three categories of synaptic connections: crucial, negligible, and variable.
• Discovered that many connections can be absent, while others exhibit broad weight distributions.
• The model predicts an ensemble of subnetworks capable of performing the same classification task.

Conclusions:

• The mean-field model offers insights into the macroscopic behavior of deep learning.
• Synaptic weight variability may encode nuisance factors, contributing to model robustness.
• Understanding distinct synaptic roles enhances transparency in deep learning decision-making.