Federated modality-specific encoders and partially personalized fusion decoder for multimodal brain tumor...

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Machine Learning

Background:

• Existing federated learning (FL) methods often fail with multimodal medical imaging due to missing data modalities among participants.
• Intermodal heterogeneity and the need for personalized models present significant challenges in collaborative medical image analysis.

Purpose of the Study:

• To propose a novel federated learning framework, FedMEPD, addressing both intermodal heterogeneity and personalization in multimodal medical image analysis.
• To develop a system capable of training a global model even when participants have incomplete imaging modalities.

Main Methods:

• FedMEPD utilizes modality-specific encoders and partially personalized multimodal fusion decoders.
• A central server with full-modal data fuses representations, while clients with incomplete modalities use cross-attention to align with global anchors.
• Personalization is achieved by dynamically adjusting decoder filters based on parameter update discrepancies.

Main Results:

• FedMEPD demonstrated superior performance on the BraTS 2018 and 2020 multimodal brain tumor segmentation benchmarks.
• The framework effectively handled intermodal heterogeneity, outperforming existing multimodal and personalized FL methods.
• Novel design components of FedMEPD were validated as effective in improving model training and performance.

Conclusions:

• FedMEPD offers a robust solution for federated learning in multimodal medical imaging, accommodating data heterogeneity.
• The proposed approach successfully balances the need for a global model with personalized insights for individual participants.
• This framework advances the applicability of federated learning to complex, real-world medical imaging scenarios.