Improving fine-tuning of self-supervised models with Contrastive Initialization.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Self-supervised learning (SSL) excels at pre-training models for downstream tasks.
• However, SSL may fail to capture semantic information, distancing same-class images in feature space.
• This hinders the subsequent fine-tuning process.

Purpose of the Study:

• To enhance semantic relations among instances for targeted downstream tasks.
• To provide improved initialization for fine-tuning self-supervised models.
• To address the issue of same-class images being distant in the learned feature space.

Main Methods:

• Propose Contrastive Initialization (COIN), a method introducing a class-aware initialization stage before fine-tuning.
• Utilize a supervised contrastive loss to increase inter-class discrepancy and intra-class compactness.
• Enrich semantics to improve the discrimination of instances across different classes.

Main Results:

• COIN significantly outperforms existing methods on multiple downstream tasks.
• Achieved accuracy improvements of 5% on ImageNet-20 and 2.57% on CIFAR100 compared to baseline.
• Demonstrates state-of-the-art performance without additional training cost.

Conclusions:

• COIN effectively enhances semantic relationships in self-supervised models.
• The proposed method provides a superior initialization strategy for fine-tuning.
• COIN sets new benchmarks for downstream task performance in computer vision.