BENCHMARKING TRANSFERABILITY OF SELF-SUPERVISED PRETRAINING FOR MULTI-ORGAN SEGMENTATION ON DIFFERENT MODALITIES

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Self-supervised learning (SSL) pretrains deep networks using unlabeled data via pretext tasks.
• Pretext task selection influences downstream task accuracy (e.g., segmentation, classification).
• Versatility of SSL features across different data modalities remains understudied.

Purpose of the Study:

• Benchmark SSL tasks for 3D medical image analysis.
• Evaluate feature versatility for multi-organ segmentation, feature reuse, and organ localization.
• Assess impact of SSL on data-limited and few-shot learning scenarios.

Main Methods:

• Utilized contrastive predictive coding, token self-distillation, and masked image modeling (MIM) for SSL pretraining.
• Employed a 3D vision transformer on 10,000 3D CT scans (1.89M images).
• Assessed performance in multi-organ segmentation, feature reuse, and organ localization using multi-head attention.

Main Results:

• SSL pretraining enhanced multi-organ segmentation accuracy in few-shot and data-limited settings for MRI and CT.
• Pretext tasks combining MIM and token self-distillation balanced local and global attention, improving segmentation.
• Feature reuse was influenced by the similarity between pretraining and fine-tuning data modalities.

Conclusions:

• SSL, particularly with combined MIM and token self-distillation, enhances segmentation performance in data-limited medical imaging.
• SSL features demonstrate versatility, but modality similarity impacts feature reuse.
• This approach shows promise for improving deep learning models in medical image analysis with limited labeled data.