ProtoSAM-2D: 2D Semantic Segment Anything Model with Mask-Level Prototype-Learning and Distillation.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Vision

Background:

• Deep learning improved medical image segmentation but requires fully supervised training on specific datasets and modalities.
• Foundation models like Segment Anything Model (SAM) offer interactive segmentation but lack crucial medical semantic understanding.
• Existing methods struggle with adaptability across diverse medical imaging scenarios and anatomical contexts.

Purpose of the Study:

• To introduce ProtoSAM-2D, an enhanced interactive segmentation framework for 2D medical images.
• To integrate semantic capabilities into SAM-based models for improved medical image analysis.
• To enable efficient categorization of anatomical structures and rapid adaptation to new classes.

Main Methods:

• Developed a novel mask-level prototype prediction mechanism for instance classification.
• Utilized learned prototypes to generate and classify feature representations of segmented instances.
• Implemented a distillation method to optimize computational efficiency of the SAM architecture and prototype classification head.

Main Results:

• ProtoSAM-2D demonstrated effectiveness in zero-shot and few-shot learning scenarios for multi-organ segmentation.
• Achieved high-quality semantic segmentation across different imaging modalities.
• Showcased efficient categorization of diverse anatomical structures and adaptability to new classes.

Conclusions:

• ProtoSAM-2D combines SAM's flexibility with prototype-based learning for adaptable semantic segmentation.
• Offers a novel solution for diverse medical imaging tasks requiring semantic understanding.
• Addresses the limitations of traditional deep learning and foundation models in specialized medical contexts.