EFM-GAN: enhanced feature modeling for multimodal MRI synthesis under missing-modality settings
Related Experiment Videos
Xin Song1, QuanQi Liu1, YiMin Wang2
1School of Computer and Communication Engineering, Northeastern University at Qinhuangdao, Qinhuangdao, Hebei, People's Republic of China.
Abstract:
Objective. Multimodal magnetic resonance imaging (MRI) synthesis has been widely investigated for recovering unavailable imaging modalities under incomplete acquisition scenarios. However, existing approaches often rely on implicit feature fusion strategies, where modality-specific appearance information and cross-modal anatomical representations are entangled in a shared latent space. Such entanglement may lead to unstable feature interaction and limit the preservation of modality-invariant structural information. Moreover, although multimodal MRI sequences are generally spatially registered, modality-specific acquisition mechanisms introduce heterogeneous feature distributions, leading to feature-level semantic inconsistency during cross-modal representation learning. Lesion-related details are also frequently underrepresented, particularly in anatomically complex regions that are difficult to reconstruct.Approach. To address these limitations, an enhanced feature modeling generative adversarial network (EFM-GAN) is proposed with an explicit parallel feature modeling framework. In EFM-GAN, parallel feature branches explicitly model modality-specific appearance and cross-modal shared representations. A bidirectional progressive alignment module is further introduced to progressively reduce feature-level semantic discrepancies between modality-specific representations before feature fusion, while preserving modality-dependent appearance information. Furthermore, a reconstruction difficulty-aware loss is designed to adaptively reweight reconstruction constraints, assigning greater emphasis to regions that are harder to reconstruct.Main results. Extensive experiments on the BRaTS2020 and BRaTS2021 datasets demonstrate that EFM-GAN achieves higher peak signal-to-noise ratio and structural similarity index than the evaluated baseline methods across multiple target modalities, with improved structural consistency and synthesis fidelity.Significance. The results demonstrate the effectiveness of explicitly modeling modality-specific and shared representations, progressively aligning heterogeneous features, and emphasizing reconstruction-difficult regions for missing-modality MRI synthesis.