Leveraging modality-guided pre-training for dual-prompt-driven multi-cancer PET-CT segmentation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Oncology

Background:

• PET-CT lesion segmentation is difficult due to lesion heterogeneity, small sizes, and physiological uptake.
• Current self-supervised methods neglect complementary PET-CT information, and multi-cancer strategies dilute cancer-specific features.
• Existing prompt-based methods lack task adaptation and sensitivity for small lesions.

Purpose of the Study:

• To develop a unified two-stage framework for improved multi-cancer PET-CT segmentation.
• To enhance cross-modal representation learning from PET-CT data.
• To improve segmentation accuracy, task adaptation, and small lesion delineation.

Main Methods:

• A modality-guided probabilistic masked autoencoder for cross-modal PET-CT representation learning.
• A dual-prompt downstream segmentation network modeling cancer-specific and shared knowledge.
• Prompt-aware heads for enhanced task adaptation and small lesion segmentation.

Main Results:

• Consistent improvements over baseline methods in multi-cancer PET-CT segmentation, with average Dice gains of 2.51% and 2.18%.
• Demonstrated generalizability on an unannotated breast cancer cohort for survival analysis.
• Achieved improved risk stratification in survival analysis.

Conclusions:

• The proposed framework effectively addresses limitations in multi-cancer PET-CT segmentation.
• The dual-prompt network enhances segmentation accuracy and adaptability for various cancer types.
• The framework shows promise for clinical applications, including survival prediction.