Stitching, Fine-Tuning, and Re-Training: A SAM-Enabled Framework for Semi-Supervised 3D Medical Image Segmentation

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Fully supervised medical image segmentation using Segment Anything Model (SAM) fine-tuning yields high performance but demands extensive annotations.
• Reducing annotation costs while maintaining segmentation accuracy is crucial for clinical applications.
• Existing SAM fine-tuning methods often overlook inter-slice contextual information in 3D medical data.

Purpose of the Study:

• To develop a semi-supervised medical image segmentation framework leveraging SAM's capabilities.
• To address the limitations of 2D slice-wise fine-tuning by incorporating 3D contextual information.
• To propose a framework that is effective, efficient, and compatible with existing semi-supervised methods.

Main Methods:

• A three-stage framework: Stitching, Fine-tuning, and Re-training (SFR) is proposed.
• A novel stitching strategy integrates 2D slices into a 3D context, mitigating mismatches.
• Stitched images are used for SAM fine-tuning to generate robust pseudo-labels, followed by 3D semi-supervised model training.

Main Results:

• The SFR framework significantly improves semi-supervised medical image segmentation performance across five datasets, especially under scarce annotation conditions.
• SFR enhances the Dice score of the Mean Teacher model from 29.68% to 74.40% on the LA dataset using only one labeled image.
• The extended SFR+ framework, incorporating selective fine-tuning and re-training via confidence estimation, further boosts performance.

Conclusions:

• The proposed SFR framework effectively reduces annotation costs for medical image segmentation while achieving high performance.
• SFR is a plug-and-play framework compatible with various semi-supervised learning methods.
• The method demonstrates significant potential for improving the efficiency and accessibility of medical image segmentation.