Biologically plausible deep learning - But how far can we go with shallow networks?

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Machine learning

Background:

• Deep neural network training via backpropagation lacks biological plausibility.
• Current research focuses on biologically plausible deep learning variants, often evaluated on MNIST accuracy.

Purpose of the Study:

• Investigate performance of biologically plausible, local learning rules in shallow networks for digit and object classification.
• Compare rate neuron models with spiking neural networks using biologically plausible learning rules.

Main Methods:

• Implemented shallow networks (one hidden, one readout layer) with fixed or unsupervised hidden layer training (PCA, Sparse Coding).
• Trained readout layer with supervised local learning rules.
• Utilized rate neurons and then spiking leaky integrate-and-fire neurons with spike timing dependent plasticity.

Main Results:

• Unsupervised learning did not outperform fixed random projections or Gabor filters for large hidden layers.
• Networks with localized receptive fields performed significantly better than all-to-all connectivity, matching backpropagation on MNIST.
• Shallow spiking networks achieved >98.2% test accuracy on MNIST, nearing backpropagation performance.

Conclusions:

• Shallow spiking networks with local learning rules offer a biologically plausible alternative to backpropagation.
• Performance is comparable to existing biologically plausible deep learning models.
• Suggests using diverse datasets beyond MNIST for future model evaluations.