STPNet: Scale-Aware Text Prompt Network for Medical Image Segmentation.

Area of Science:

• Medical Image Analysis
• Computer Vision
• Artificial Intelligence

Background:

• Accurate medical image segmentation is crucial for diagnosis but challenged by lesion uncertainty.
• Traditional methods relying on visual features alone have limitations in handling complex lesion characteristics.

Purpose of the Study:

• To introduce STPNet, a Scale-aware Text Prompt Network, for enhanced medical image segmentation.
• To leverage vision-language modeling and multi-scale textual descriptions to improve lesion localization and segmentation accuracy.

Main Methods:

• Developed STPNet, a Scale-aware Text Prompt Network, utilizing vision-language modeling.
• Employed retrieval-segmentation joint learning to bridge visual and linguistic modalities.
• Integrated multi-scale textual descriptions for lesion guidance and retrieval from a medical text repository during training.

Main Results:

• STPNet demonstrated superior performance compared to state-of-the-art segmentation methods on COVID-Xray, COVID-CT, and Kvasir-SEG datasets.
• The vision-language approach effectively incorporated textual semantic knowledge, enhancing segmentation accuracy.
• The model successfully eliminated the need for text input during inference while retaining cross-modal learning benefits.

Conclusions:

• STPNet offers a significant advancement in medical image segmentation by effectively integrating textual semantic information.
• The proposed vision-language approach shows great potential for improving diagnostic accuracy in medical image analysis.
• The method's ability to leverage a specialized medical text repository during training enhances its robustness and generalizability.