Fine-Grained Perception for Fundus and Prostate Medical Image Segmentation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Deep learning models struggle with medical image segmentation across diverse domains due to distribution shifts.
• The Segment Anything Model (SAM) shows promise for generalization but lacks domain-specific medical imaging knowledge.
• SAM's performance is limited by unreliable prompts and patch-wise inference, affecting anatomical detail capture.

Purpose of the Study:

• To develop a novel model, MedFineSAM, that enhances SAM's capabilities for medical image segmentation.
• To improve the generalization of SAM in unseen medical imaging domains.
• To address limitations in fine-grained structural knowledge, prompt reliability, and structural continuity in SAM for medical applications.

Main Methods:

• Proposed MedFineSAM integrates three modules: shared fine-grained structural enhancement, a prompt gating mechanism, and structural continuity diffusion in the frequency domain (SCFD).
• Shared fine-grained structural enhancement utilizes a structural dictionary to extract and enhance features.
• Prompt gating dynamically adjusts prompt weights based on confidence, while SCFD ensures structural continuity during decoding.

Main Results:

• MedFineSAM demonstrated superior generalization performance on fundus and prostate MRI benchmarks.
• The model effectively addresses limitations of the base SAM in medical image segmentation.
• Experiments confirmed the efficacy of the proposed modules in enhancing segmentation accuracy and continuity.

Conclusions:

• MedFineSAM offers a significant advancement for single-source domain generalization in medical image segmentation.
• The proposed approach provides new insights into adapting large foundation models like SAM for specialized medical tasks.
• MedFineSAM shows potential for improving the reliability and accuracy of automated medical image analysis.