Prompt Learning With Bounding Box Constraints for Medical Image Segmentation

Area of Science:

• Medical Imaging
• Computer Vision
• Machine Learning

Background:

• Pixel-wise annotations for medical image segmentation are time-consuming and expensive.
• Weakly supervised methods using bounding boxes offer a more efficient alternative.
• Vision foundation models show promise in segmentation with prompt-based learning.

Purpose of the Study:

• To develop a novel framework combining foundation models with weakly supervised segmentation.
• To automate prompt generation for foundation models using only bounding box annotations.
• To reduce the burden of manual annotation in medical image segmentation.

Main Methods:

• A novel framework integrating foundation models with weakly supervised segmentation.
• Automated prompt generation for foundation models utilizing bounding box annotations.
• An optimization scheme combining box annotation constraints with pseudo-labels from prompted foundation models.

Main Results:

• The proposed weakly supervised method achieved an average Dice score of 84.90% in a limited data setting.
• The approach outperformed existing fully-supervised and weakly-supervised methods.
• Demonstrated effectiveness across multi-modal datasets.

Conclusions:

• The developed framework successfully leverages foundation models for efficient medical image segmentation.
• Automated prompt generation with bounding boxes significantly reduces annotation effort.
• This method offers a practical and high-performing solution for medical image segmentation challenges.