A weakly supervised model for incomplete multimodal MRI synthesis with tumor-aware approach

Magnetic resonance images (MRIs) are a valuable tool in the study of brain tumors, and multimodal sequences provide unique insights into different aspects of brain tumors. However, in clinical practice, missing modalities are often encountered due to various factors. This makes it difficult to obtain comprehensive and reliable information related to brain tumors.

The purpose of this work is to develop an algorithm for the synthesis of missing MRI modality with high precision, and to center on generating accurate tumor-related information to offer more data for clinical diagnosis.

A novel weakly supervised MRI synthesis model named TAM-DAM-GAN has been proposed, which integrates tumor-aware and detail adjustment mechanisms to enhance the quality of tumor generation. The tumor-aware mechanism leverages weak label information to guide the network to classify images based on crucial information in local structures, thereby compelling the generative network to identify and highlight the learning of local tumor regions. The detail adjustment mechanism utilizes a discriminator to create attention maps at the pixel level in real-time. These maps are then used to modify the loss weight, which in turn adjusts the details that are generated.

Generation quality of four tasks (T1-to-T2, T2-to-T1, T1-to-FLAIR, and FLAIR-to-T1) was evaluated. Experiments on the BRATS2015 dataset show that the proposed approach is superior in both qualitative and quantitative measures. Taking FLAIR-to-T1 as an example, TAM-DAM-GAN improves PSNR of tumor region from 18.556 to 20.576 compared to baseline. Also, using real FLAIR data with generated T1 data boosts tumor segmentation accuracy by 10% compared to using only real FLAIR data.

This finding will be conducive to enhancing the accuracy of cross-modality synthesis in incomplete multimodal MRI, especially for tumor regions, thereby providing more dependable and comprehensive data for clinical diagnosis and scientific research.