A Text-Image Network for Isocitrate Dehydrogenase(IDH) Mutation Status Prediction in Glioma Diagnosis Using...

Brain glioma is a very serious disease and the mutation status of isocitrate dehydrogenase (IDH) is an important factor in its diagnosis. There is a clear link between the prognosis of glioma and the IDH mutation, and knowing the status of the IDH mutation helps physicians plan treatment strategies. However, current methods for detecting IDH mutations are costly and not always practical. Clinically, there is a recognized relationship between magnetic resonance images (MRI) images and the status of the IDH mutation. In recent years, many machine learning methods have been developed to predict the IDH mutation using magnetic resonance images. Most of these studies focus solely on the modality of the magnetic resonance image and ignore the text in radiology reports, which contains valuable diagnostic information. This limits the benefits of a multimodal approach in clinical diagnosis. To address this gap, our study proposes a multimodal deep learning model that uses 3D MRI images and text reports to predict IDH mutation status. We evaluated our method using the BraTS20 challenge dataset, with the text modality annotated by the First Affiliated Hospital of Zhengzhou University in China. Compared with state-of-the-art methods, our approach improves the accuracy of predicting IDH mutation status by 4%, demonstrating better overall performance.