Radiosensitivity Prediction of Tumor Patient Based on Deep Fusion of Pathological Images and Genomics

The radiosensitivity of cancer patients determines the efficacy of radiotherapy, and patients with low radiosensitivity cannot benefit from radiotherapy. Therefore, accurately predicting radiosensitivity before treatment is essential for personalized and precise radiotherapy. However, most existing studies rely solely on genomic and clinical features, neglecting the tumor microenvironmental information embedded in histopathological images, which limits prediction accuracy. To address this issue, we propose Resfusion, a deep multimodal fusion framework that integrates patient-level gene expression profiles, clinical records, and histopathological images for tumor radiosensitivity prediction. Specifically, the pre-trained large-scale pathology model is used as an image encoder to extract global representations from whole-slide pathological image. Radiosensitivity-related genes are selected using an autoencoder combined with univariate Cox regression, while clinically relevant variables are manually curated. The three modalities are first concatenated and then refined through a self-attention-based module, which captures inter-feature dependencies within the fused representation and highlights complementary information across modalities. The model was evaluated using five-fold cross-validation on two common tumor datasets suitable for radiotherapy: the Breast Invasive Carcinoma (BRCA) dataset (282 patients in total, with each fold partitioned into 226 training samples and 56 validation samples) and the Head and Neck Squamous Cell Carcinoma (HNSC) dataset (200 patients in total, with each fold partitioned into 161 training samples and 39 validation samples). The average AUC values obtained from the five-fold cross-validation reached 76.83% and 79.49%, respectively. Experimental results demonstrate that the Resfusion model significantly outperforms unimodal methods and existing multimodal fusion methods, verifying its effectiveness in predicting the radiosensitivity of tumor patients.