Seed the Views: Hierarchical Semantic Alignment for Contrastive Representation Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Self-supervised learning, particularly instance discrimination and contrastive learning, has advanced representation learning.
• Conventional contrastive learning methods do not explicitly model semantic relationships between similar samples.

Purpose of the Study:

• To propose a general module, termed CSML, that explicitly incorporates semantic similarity into contrastive learning.
• To enhance representation learning by modeling invariance to semantically similar images hierarchically.

Main Methods:

• Expanding image views to Cross-Samples (via constrained data mixing) and Multi-Levels (at intermediate network layers).
• Modifying contrastive loss to accommodate multiple positives per anchor and pull semantically similar images together across network layers.

Main Results:

• CSML integrates multi-level representations across samples robustly and improves performance when applied to existing contrastive methods.
• Achieved 76.6% top-1 accuracy with linear evaluation (ResNet-50 backbone) and state-of-the-art results with limited labels (66.7% with 1% and 75.1% with 10%).

Conclusions:

• CSML offers a significant advancement in self-supervised representation learning by effectively leveraging semantic similarity.
• The method demonstrates broad applicability and sets new benchmarks in performance, particularly in low-label regimes.