ProtoSAM for automated one shot medical image segmentation using foundational models.

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• One-shot medical image segmentation is crucial for scenarios with limited labeled data.
• Rapid adaptation to new data classes and sites is a significant challenge in medical image analysis.

Purpose of the Study:

• To introduce ProtoSAM, a novel, automated framework for one-shot medical image segmentation.
• To leverage Prototypical networks and the Segment Anything Model (SAM) for efficient segmentation adaptation.
• To demonstrate state-of-the-art performance on diverse medical imaging datasets.

Main Methods:

• ProtoSAM combines Prototypical networks (ALPnet with DINOv2 encoder) for initial mask generation.
• Extracted prompts (points, bounding boxes) from the initial mask are fed into the Segment Anything Model (SAM).
• The framework is validated on CT, MRI, and endoscopy image datasets.

Main Results:

• ProtoSAM achieves state-of-the-art results across various medical imaging segmentation tasks.
• An untrained ProtoSAM matches or surpasses existing one-shot trained methods.
• Self-supervised finetuning of the encoder further enhances ProtoSAM's performance.

Conclusions:

• ProtoSAM offers a powerful and adaptable solution for one-shot medical image segmentation.
• The framework demonstrates significant potential for applications with scarce labeled data.
• The availability of the code facilitates further research and development in this area.