Contrastive Self-Supervised Learning As Neural Manifold Packing

Area of Science:

• Computer Vision
• Machine Learning
• Computational Neuroscience

Background:

• Contrastive self-supervised learning is prevalent in vision tasks.
• Neural manifolds in the brain organize stimulus responses, crucial for classification.
• Separating these manifolds is key, analogous to packing problems.

Purpose of the Study:

• Introduce Contrastive Learning As Manifold Packing (CLAMP), a self-supervised framework.
• Reframe representation learning as a manifold packing problem.
• Bridge insights from physics, neural science, and machine learning.

Main Methods:

• Developed a novel loss function inspired by repulsive particle systems in physics.
• Treated each class as sub-manifolds of augmented image views.
• Dynamically optimized sub-manifold positions and sizes using a packing loss gradient.

Main Results:

• Achieved competitive performance against state-of-the-art self-supervised models under linear evaluation.
• Demonstrated interpretable dynamics in the embedding space mirroring jamming physics.
• Observed natural emergence and effective separation of neural manifolds for different categories.

Conclusions:

• CLAMP offers a new perspective on representation learning by treating it as a manifold packing problem.
• The framework effectively separates neural manifolds, leading to improved classification.
• CLAMP shows promise in integrating concepts from physics, neuroscience, and machine learning.