Hebbian dreaming for small datasets.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Artificial Intelligence

Background:

• The Hebbian paradigm is foundational for neural networks.
• Artificial neural networks often require vast datasets for training.
• Biological systems exhibit efficient learning with less data.

Purpose of the Study:

• To investigate the data efficiency of the dreaming Hopfield model.
• To compare the information requirements of the dreaming Hopfield model with the standard Hopfield model.
• To analyze the computational skills and performance of the dreaming Hopfield model.

Main Methods:

• The study employs the dreaming Hopfield model, incorporating both online (awake) and offline (sleeping) learning mechanisms.
• Minimal information thresholds for generalization were assessed on synthetic and standard datasets (MNIST, Fashion-MNIST, Olivetti).
• The model's cost function was represented as a standard machine learning loss function for theoretical and computational analysis.

Main Results:

• The dreaming Hopfield model achieved performance comparable to the standard Hopfield model while requiring up to 90% less data.
• Offline sleeping mechanisms were shown to enhance storage capacity, approaching theoretical limits.
• A quantitative analysis revealed the model's capabilities as a function of its control parameters, confirming theoretical predictions.

Conclusions:

• The dreaming Hopfield model demonstrates significant data reduction, suggesting biological 'sleep' mechanisms are key to efficient learning.
• This model serves as an associative memory for pattern recognition, learning online and generalizing effectively.
• The findings advocate for sustainable AI development, particularly in data-sparse environments.