PROMISE: PROMPT-DRIVEN 3D MEDICAL IMAGE SEGMENTATION USING PRETRAINED IMAGE FOUNDATION MODELS.

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Medical imaging faces challenges in data acquisition and labeling.
• Transfer learning from natural to medical image domains is a promising strategy.
• Barriers include contrast discrepancies, anatomical variability, and 2D-to-3D adaptation.

Purpose of the Study:

• To propose ProMISe, a prompt-driven 3D medical image segmentation model.
• To leverage knowledge from a pretrained 2D foundation model (Segment Anything Model - SAM) for 3D tasks.
• To overcome domain shift challenges in medical image analysis.

Main Methods:

• Utilizes a pretrained vision transformer from SAM with lightweight adapters for 3D context extraction.
• Employs a hybrid network with complementary encoders for robust feature extraction.
• Incorporates a boundary-aware loss function for precise segmentation boundaries.
• Uses a single point prompt for efficient segmentation.

Main Results:

• ProMISe achieves superior performance on colon and pancreas tumor segmentation datasets.
• Outperforms state-of-the-art methods in both prompt-engineered and non-prompt-engineered settings.
• Demonstrates effective adaptation of 2D models for 3D medical segmentation tasks.

Conclusions:

• ProMISe offers a viable and effective solution for 3D medical image segmentation.
• The prompt-driven approach successfully bridges the domain gap between natural and medical images.
• The model's ability to extract 3D context from 2D pretrained models is a key advancement.