Storage and Learning Phase Transitions in the Random-Features Hopfield Model

Area of Science:

• Computational Neuroscience
• Statistical Physics
• Machine Learning

Background:

• The Hopfield model is a foundational neural network model studied across multiple disciplines.
• Existing models primarily focus on pattern retrieval, lacking mechanisms for unsupervised feature inference.

Purpose of the Study:

• To propose and analyze a generalized Hopfield model incorporating random features.
• To investigate the model's behavior and phase transitions in a large-scale limit.

Main Methods:

• Introduced the random-features Hopfield model, generating patterns via random projections and nonlinearities.
• Employed the replica method from statistical physics to derive the model's phase diagram.
• Analyzed the model in the limit of large patterns, network size, and latent dimension.

Main Results:

• Identified the standard retrieval phase where original patterns are recovered.
• Discovered a novel 'learning phase transition' where the model recovers latent features.
• Demonstrated unsupervised feature inference without explicit programming.

Conclusions:

• The random-features Hopfield model extends traditional frameworks by enabling unsupervised feature learning.
• The 'learning phase transition' offers new insights into how neural networks can infer underlying data structures.
• This work bridges concepts from machine learning's manifold hypothesis with statistical physics models of neural networks.