Weak Augmentation Guided Relational Self-Supervised Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Self-supervised Learning (SSL) excels at learning visual representations from unlabeled data.
• Current SSL methods primarily focus on instance-level features, neglecting inter-instance relationships.

Purpose of the Study:

• To introduce a novel Self-supervised Learning paradigm, Relational Self-supervised Learning (ReSSL), that models relationships between different instances.
• To enhance visual representation learning by capturing inter-instance dynamics.

Main Methods:

• Developed a ReSSL framework utilizing a sharpened distribution of pairwise similarities as a relation metric.
• Employed weak augmentations for reliable relation representation and a momentum strategy for efficiency.
• Incorporated an asymmetric predictor head and an InfoNCE warm-up strategy to improve robustness and performance.

Main Results:

• The proposed ReSSL framework demonstrated superior performance compared to state-of-the-art methods.
• Significant improvements were observed across various network architectures, including lightweight models like EfficientNet and MobileNet.

Conclusions:

• ReSSL offers a powerful new paradigm for self-supervised visual representation learning.
• Modeling inter-instance relationships is crucial for advancing SSL techniques.