How Well Do Self-Supervised Models Transfer to Medical Imaging?

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Vision

Background:

• Self-supervised learning (SSL) has shown promise in transferring knowledge between similar medical imaging datasets.
• A large-scale comparison of different SSL models' transferability across diverse medical imaging datasets is lacking.
• Understanding model generalizability is crucial for effective deployment in clinical settings.

Purpose of the Study:

• To compare the generalizability of seven self-supervised models against supervised baselines across eight medical datasets.
• To evaluate the performance of in-domain versus general-domain pre-trained self-supervised models on medical classification tasks.
• To investigate the impact of pre-training strategies on feature representation and model transferability in medical imaging.

Main Methods:

• Seven self-supervised models were evaluated, including two trained on in-domain medical data.
• Performance was compared against supervised learning baselines.
• Models were tested across eight diverse medical imaging datasets for classification tasks.

Main Results:

• ImageNet-pretrained self-supervised models demonstrated superior generalizability, achieving up to 10% higher scores on medical classification tasks compared to supervised models.
• In-domain pre-trained models significantly outperformed others (over 20%) on their specific training tasks but showed reduced accuracy on other datasets.
• Analysis of feature representations suggested that in-domain models may overfit to specific image regions, limiting broader transferability.

Conclusions:

• Self-supervised learning models, particularly those pre-trained on large general datasets like ImageNet, offer enhanced generalizability for medical imaging tasks.
• While in-domain pre-training boosts performance on specific medical tasks, it compromises broad applicability, highlighting a trade-off in model specialization.
• Future research should focus on developing SSL strategies that balance domain-specific feature learning with robust generalizability across diverse medical imaging applications.