Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging

Area of Science:

• Artificial Intelligence
• Computer Vision
• Digital Pathology

Background:

• The Segment Anything Model (SAM) is a powerful foundation model for image segmentation.
• It is trained on over 1 billion masks and supports zero-shot segmentation using various prompts.
• SAM's capabilities are attractive for medical image analysis, particularly in digital pathology due to limited training data.

Purpose of the Study:

• To evaluate the zero-shot segmentation performance of the SAM model on whole slide imaging (WSI).
• To assess SAM's effectiveness on representative digital pathology tasks: tumor segmentation, non-tumor tissue segmentation, and cell nuclei segmentation.

Main Methods:

• Utilized the Segment Anything Model (SAM) in a zero-shot setting.
• Applied SAM to whole slide images (WSIs) for segmentation tasks.
• Evaluated performance on tumor, non-tumor tissue, and cell nuclei segmentation.

Main Results:

• SAM demonstrated remarkable segmentation performance for large, connected objects.
• The model did not consistently achieve satisfactory results for dense instance object segmentation, even with multiple prompts.
• Identified limitations include image resolution, multiple scales, prompt selection, and the need for model fine-tuning.

Conclusions:

• The zero-shot SAM model is effective for segmenting large structures in digital pathology but requires improvement for dense object segmentation.
• Future work should focus on few-shot fine-tuning with downstream pathological segmentation data to enhance performance on complex tasks.