Contrastive Learning With Stronger Augmentations.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Representation learning has advanced with contrastive learning methods.
• Current methods rely on data augmentations that preserve instance identity, limiting exploration of novel patterns.
• Direct contrastive learning with strong augmentations is often ineffective.

Purpose of the Study:

• To propose a general framework, Contrastive Learning with Stronger Augmentations (CLSA), to enhance representation learning.
• To leverage information from strongly augmented images that are typically ignored.
• To improve the effectiveness of contrastive learning by incorporating stronger data transformations.

Main Methods:

• Introduced a novel framework, Contrastive Learning with Stronger Augmentations (CLSA).
• Utilized distribution divergence between weakly and strongly augmented images within a representation bank.
• Supervised retrieval of strongly augmented queries from a pool of instances.

Main Results:

• CLSA significantly boosts performance by incorporating information from strongly augmented images.
• Achieved a top-1 accuracy of 76.2% on ImageNet using a standard ResNet-50 architecture.
• Performance is comparable to supervised results (76.5% accuracy).

Conclusions:

• The proposed CLSA framework effectively complements existing contrastive learning approaches.
• Stronger augmentations, when properly utilized, provide valuable information for representation learning.
• This method offers a promising direction for advancing self-supervised learning techniques.