Outlier-Aware Contrastive Learning.

Area of Science:

• Machine Learning
• Computer Vision

Background:

• Contrastive learning aims to create discriminative embedding spaces.
• Sampling bias, caused by mislabeled similar or dissimilar samples, degrades contrastive learning performance.
• Out-of-distribution (OOD) detection can mitigate this bias by identifying and masking false negatives.

Purpose of the Study:

• To develop an outlier-aware contrastive learning method that effectively debiases models without prior knowledge of OOD samples.
• To improve the fidelity and classification performance of contrastive learning models.

Main Methods:

• Proposed a novel approach using generated and augmented samples near the in-distribution (ID) and OOD boundary.
• Synthesized Gaussian embeddings for these samples to mimic OOD behaviors.
• Trained an OOD detector and a contrast model jointly using ID and synthesized OOD samples.

Main Results:

• Demonstrated the effectiveness of the proposed outlier-aware contrastive learning method.
• Showcased the ability to debias contrast models by addressing sampling bias.
• Validated the merit of using synthesized OOD samples for detector training.

Conclusions:

• The proposed method successfully enhances contrastive learning by effectively handling sampling bias through outlier detection.
• This approach offers a practical solution for real-world scenarios where OOD sample knowledge is unavailable.
• The study highlights the potential of generative techniques for improving robust representation learning.