Eigen value based loss function for training attractors in iterated autoencoders.

Area of Science:

• Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• The human brain's ability to handle input variations and act as an attractor for memory is a key area of neuroscience research.
• Existing models for associative memory often struggle with deep local minima and parameter efficiency.

Purpose of the Study:

• To propose a new method for creating attractors during the training of iterated autoencoders.
• To enhance the brain-inspired associative memory capabilities of artificial neural networks.

Main Methods:

• A novel loss function is introduced to decrease the absolute real part of Eigen values while maintaining reconstruction accuracy.
• A fully connected iterated autoencoder architecture is employed, utilizing a layer-by-layer pre-training approach to overcome local minima.
• The model's performance is evaluated on the MNIST dataset.

Main Results:

• The proposed model achieved 59.98% retrieval accuracy on MNIST test samples, surpassing Dense Associative Memory (DAM).
• It demonstrated superior performance over the Overparameterized Autoencoder (OAE) in learning attractors with fewer network parameters.
• The model exhibited significant robustness against corrupted training samples compared to a baseline autoencoder.

Conclusions:

• The developed iterated autoencoder with a novel loss function effectively creates attractors for improved memory retrieval.
• This brain-inspired approach offers a more robust and parameter-efficient method for building associative memory systems.