All-in-SAM: from Weak Annotation to Pixel-wise Nuclei Segmentation with Prompt-based Finetuning

Area of Science:

• Computer Vision
• Medical Image Analysis
• Artificial Intelligence

Background:

• The Segment Anything Model (SAM) offers powerful zero-shot segmentation capabilities.
• Manual prompting in SAM is resource-intensive for biomedical applications.
• Efficient segmentation is crucial for advancing medical image analysis.

Purpose of the Study:

• To introduce an automated pipeline, all-in-SAM, for biomedical image segmentation using SAM.
• To eliminate the need for manual prompts during SAM inference.
• To enhance the efficiency and performance of AI development workflows in medical imaging.

Main Methods:

• Utilized SAM for automatic pixel-level annotation generation from weak prompts (points, bounding boxes).
• Employed generated annotations to finetune the SAM model.
• Evaluated the pipeline on a nuclei segmentation task using the Monuseg dataset.

Main Results:

• The all-in-SAM pipeline outperformed state-of-the-art methods in nuclei segmentation.
• Finetuning SAM with weak/few annotations achieved performance competitive with strong pixelwise annotations.
• Demonstrated the effectiveness of automated annotation and finetuning for SAM.

Conclusions:

• The all-in-SAM pipeline significantly improves efficiency in biomedical image segmentation.
• Automated annotation and finetuning with SAM offer a viable alternative to manual segmentation.
• This approach holds promise for accelerating AI development in medical imaging.