Bidirectional Perceptual Multimodal Interaction Network Based on Contrastive Learning for Breast Cancer pCR...

Early and accurate prediction of pathological complete response (pCR) after neoadjuvant chemotherapy (NAC) is vital for personalized breast cancer treatment. However, existing deep learning methods are hampered by tumor heterogeneity and semantic misalignment between high-dimensional dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) and low-dimensional clinical data, which limits pCR prediction performance and generalization. This study addresses these challenges via a novel multimodal network.

We propose a Bidirectional Perceptual Multimodal Interaction Network (BPMINet) based on contrastive learning. BPMINet integrates pre-NAC DCE-MRI and clinical information through three core components: (1) we propose a bidirectional cross-modal attention (BiCMA) fusion mechanism to resolve semantic misalignment and facilitate effective multimodal feature fusion; (2) we design a multimodal contrast-aware feature enhancement (MCFE) module as a key component tightly integrated into the pCR-oriented contrastive learning framework, which serves to boost discriminative power for pCR prediction and improve generalization performance on hard-to-classify samples; (3) we adopt a dual-loss strategy to enable the collaborative optimization of discriminative feature representation and pCR prediction performance.

On two publicly available multicenter datasets, BPMINet outperformed all comparative methods across seven evaluation metrics: specifically, it surpassed the top-performing baseline by 5.17% in AUC and 5.24% in accuracy on the MAMA-MIA dataset. More notably, it achieved substantially larger gains of 11.72% in AUC and 7.38% in accuracy on the ISPY1 dataset.

BPMINet achieves optimal pCR prediction performance, confirming its superiority and strong generalization ability for multimodal breast cancer pCR prediction.