Radiation Pneumonitis Prediction Using Dual-Modal Data Fusion Based on Med3D Transfer Network

Radiation pneumonitis (RP) is an inflammatory reaction in the lungs caused by radiation therapy. The etiology of RP is complex and varied, making it challenging to establish accurate predictive models for RP. The aim of this study is to develop a dual-modal prediction model for RP using pre-radiotherapy CT images of the patient's lungs, along with clinical and dose data. In this paper, an RP prediction model utilizing dual-modal data is proposed. Firstly, for CT image data, the Med3D transfer network, trained on a large-scale medical dataset, is employed to extract CT image features. To adapt the transfer network for the RP task, the last convolutional block, which incorporates the 3D channel attention mechanism, is trained and saved as the optimal model for extracting deep features once training stabilizes. Subsequently, an autoencoder (AE) is employed to compress the deep features to reduce their dimensionality. Secondly, for clinical and dose features, univariate analysis and lasso regression are used to screen the features. Finally, the two groups of features are multiplied by their respective adaptive rates to achieve data fusion, and then input into the binary classification model for training and prediction. The KAN classifier with adaptive feature fusion demonstrates superior performance, achieving precision rates of 74% and 76%, a recall rate of 73%, and an AUC of 86%. These results surpass those obtained from single-modality approaches. The experimental results on a dataset of 117 chest cancer patients receiving radiotherapy show that the dual-modal data fusion model can predict RP more effectively than single-modality models.