Unsupervised learning on spontaneous retinal activity leads to efficient neural representation geometry.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Developing mammalian retinas exhibit spontaneous, correlated neural activity known as retinal waves before vision onset.
• Retinal waves are hypothesized to play a crucial role in shaping early sensory representations and neural circuit development.
• Understanding this pre-visual developmental stage can inform artificial intelligence by mimicking biological learning principles.

Approach:

• Utilized unsupervised contrastive learning (SimCLR) to pre-train a ResNet-18 neural network.
• Employed both simulated and experimental movies of retinal waves as pre-training data.
• Evaluated the network's performance on image classification tasks, focusing on object invariance and feature representation.

Key Points:

• Pre-training on retinal wave data significantly improved performance on tasks requiring invariance to spatial translation.
• A slight performance boost was observed on more complex image classification tasks.
• The benefits were evident on natural images not included in the pre-training dataset.
• Retinal wave characteristics were shown to facilitate the formation of separable feature representations in neural networks.

Conclusions:

• The spatiotemporal properties of retinal waves prepare neural networks for effective higher-order feature extraction.
• This biologically inspired pre-training approach offers a novel method for enhancing AI model robustness and learning capabilities.
• Findings suggest a direct link between early visual system activity patterns and the development of robust visual representations.