Multimodal Fusion of 3D CT and Pathological Images for Gastric Cancer Recurrence Prediction

Gastric cancer recurrence severely impacts postoperative outcomes, and accurate prediction is crucial for personalized management. 3D CT images (macroscopic lesion context) and Whole Slide Images (WSIs, microscopic histopathological details) offer complementary information, but effective fusion is hindered by feature dimensionality disparity and lack of robust integration strategies. Existing models show suboptimal performance due to inadequate multimodal fusion, resulting in unreliable risk assessments that cannot be clinically applied for personalized therapy.

To preoperatively identify high-risk patients and enable personalized postoperative follow-up and treatment stratification to assist clinicians, this study adopts a fusion strategy combining multi-stage attention and co-attention mechanisms to achieve efficient integration of Whole Slide Images (WSIs) and CT images, thereby providing an accurate, robust and generalizable solution for gastric cancer recurrence prediction.

This retrospective multi-center study included three datasets: a primary cohort (646 patients, Zhongshan Hospital) and two independent test sets (160 patients, Zhongshan Hospital Xiamen Branch; 140 patients, Taicang TCM Hospital). CT features were extracted using DSMAGNet integrated with the iSAFF module and GateNetwork, while Whole Slide Image (WSI) features were derived via multi-stage attention-based dimensionality reduction (MSAT). Finally, multimodal fusion of the two types of features was accomplished through a co-attention mechanism.

The framework achieved an AUC of 83.4% on the primary dataset, outperforming 11 comparative methods by up to 4.2%. On external test sets, it showed superior performance with AUC improvements of 4.63% and 3.91% vs. the next-best methods. Ablation studies confirmed the effectiveness of DSMAGNet and MSAT.

The multimodal framework enables accurate, interpretable, and generalizable gastric cancer recurrence prediction by integrating WSI and CT images. It aids preoperative identification of high-risk patients, supporting personalized postoperative follow-up and treatment stratification to improve long-term outcomes.