BTSSPro: Prompt-Guided Multimodal Co-Learning for Breast Cancer Tumor Segmentation and Survival Prediction

Area of Science:

• Medical Imaging Analysis
• Artificial Intelligence in Oncology
• Biomedical Informatics

Background:

• Early breast cancer detection via medical imaging improves survival rates.
• Current methods struggle with multi-modal data, leading to semantic disparities and reduced prognostic accuracy.
• Sharing parameters between tumor segmentation and survival prediction is clinically beneficial.

Purpose of the Study:

• To propose BTSSPro, a novel Prompt-guided multi-modal co-learning framework for concurrent breast cancer tumor segmentation and survival prediction.
• To enhance prognostic accuracy by addressing challenges in multi-modal data integration and semantic disparities.
• To leverage shared parameters between segmentation and prediction tasks.

Main Methods:

• Utilized shared dual attention (SDA) blocks for extracting tumor-specific discriminative features from breast MR images.
• Employed a guided fusion module (GFM) to integrate Electronic Health Record (EHR) vectors with imaging features.
• Introduced a feature harmonic unit (FHU) to synchronize transformer encoder and CNN decoder, minimizing semantic differences.

Main Results:

• BTSSPro achieved a C-index of 0.968 and Dice score of 0.715 on the Breast MRI-NACT-Pilot dataset.
• The model attained a C-index of 0.807 and Dice score of 0.791 on the ISPY1 dataset.
• Demonstrated superior performance compared to existing state-of-the-art methods.

Conclusions:

• BTSSPro effectively integrates multi-modal data for improved breast cancer tumor segmentation and survival prediction.
• The proposed framework enhances prognostic accuracy by reducing semantic disparities and leveraging task correlations.
• This approach offers a promising advancement in AI-driven cancer diagnostics and prognostics.