Competitive learning to generate sparse representations for associative memory.

Area of Science:

• Computational Neuroscience
• Machine Learning
• Image Processing

Background:

• Hebbian learning and neural assemblies are foundational brain principles.
• Palm's model uses Willshaw associative memory but requires extremely sparse data.
• Real-world data sparsity is a challenge for applying such models.

Purpose of the Study:

• To develop a biologically-constrained network for encoding images into sparse representations.
• To enable the application of Willshaw associative memory to real-world data.
• To address the need for efficient data encoding aligned with Barlow's principle.

Main Methods:

• A biologically-constrained network with neuron groups specializing in local receptive fields.
• A competitive learning scheme for network training.
• Auto- and hetero-association experiments on visual datasets.

Main Results:

• The proposed network effectively encodes images into sparse representations suitable for Willshaw associative memory.
• The network outperforms existing sparse coding baselines.
• Performance closely approaches that of optimal random codes.

Conclusions:

• The developed network provides a viable solution for encoding real-world data for sparse associative memory.
• This approach bridges the gap between theoretical models and practical applications in neuroscience and AI.
• The findings support the development of more biologically plausible and efficient AI systems.