Deep Residual Autoencoders for Expectation Maximization-Inspired Dictionary Learning.

Area of Science:

• Machine Learning
• Computational Neuroscience
• Signal Processing

Background:

• Dictionary learning is crucial for signal representation.
• Existing methods often lack efficiency and direct neural network integration.
• Bridging dictionary learning and neural networks offers new computational possibilities.

Purpose of the Study:

• Introduce a novel neural network architecture, the constrained recurrent sparse autoencoder (CRsAE).
• Establish a direct link between dictionary learning and neural networks.
• Develop an efficient method for simultaneous dictionary and regularization parameter training.

Main Methods:

• Leveraged the expectation-maximization (EM) algorithm interpretation for dictionary learning.
• Developed autoencoders with a forward pass approximating E-step sufficient statistics using FISTA.
• Implemented M-step via two-stage backpropagation for dictionary and regularization parameter updates.

Main Results:

• CRsAE learns Gabor-like filters in image denoising, outperforming conventional bias learning.
• Achieved a 900x speedup in identifying spike times from brain recordings.
• Learned realistic spike templates from neural data.

Conclusions:

• CRsAE successfully integrates dictionary learning principles into neural networks.
• The EM-inspired approach for learning biases is superior to conventional methods.
• CRsAE offers significant computational advantages for signal processing and neuroscience applications.