Text-Image Co-Alignment for Weakly Supervised Polyp Segmentation

Area of Science:

• Medical image analysis
• Computer vision
• Artificial intelligence in healthcare

Background:

• Fully supervised polyp segmentation requires expensive pixel-level annotations.
• Existing semi- and weakly supervised methods still need partial mask supervision.
• Text-supervised segmentation offers a promising alternative but faces challenges in precise phrase-region grounding for polyps.

Purpose of the Study:

• To develop a text-supervised framework for accurate polyp segmentation.
• To leverage large language models (LLMs) for generating weak supervision from clinical descriptions.
• To address the challenge of grounding instance-specific phrases to correct polyp regions.

Main Methods:

• Proposed Text-Image Co-Alignment (TICoA) framework for text-supervised polyp segmentation.
• Utilized LLM-generated structured clinical descriptions as weak supervision.
• Employed contrastive learning for explicit phrase-region association and a State-Space Model (Mamba) with a Mamba Fusion module and Bi-Dimension Fusion (BiDF) for efficient modeling of long-range dependencies and cross-modal interaction.

Main Results:

• TICoA demonstrates competitive performance compared to state-of-the-art weakly supervised methods on polyp segmentation tasks.
• Validation on skin lesion segmentation datasets further supports the framework's effectiveness.
• The proposed Mamba-based architecture efficiently handles long-range dependencies and cross-modal fusion.

Conclusions:

• Text-Image Co-Alignment (TICoA) provides an effective text-supervised approach for polyp segmentation.
• The framework successfully grounds textual descriptions to image regions, reducing reliance on manual annotations.
• TICoA shows promise for advancing automated analysis in medical imaging, with potential applications beyond polyp segmentation.