Multimodal Federated Learning: A Survey.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• Federated learning (FL) enables collaborative training on distributed data while preserving privacy.
• Existing FL research primarily addresses unimodal data and the non-IID (non-identical distribution) challenge.
• Real-world data is often multimodal, yet multimodal federated learning (MFL) remains underexplored.

Purpose of the Study:

• To highlight the significance of multimodal federated learning (MFL).
• To provide a comprehensive literature review of state-of-the-art MFL methods.
• To categorize MFL approaches and identify application tasks and benchmarks.

Main Methods:

• Literature review of existing multimodal federated learning studies.
• Categorization of MFL into congruent and incongruent types based on modal availability across clients.
• Investigation of application tasks and benchmarks relevant to MFL.

Main Results:

• Identified a gap in research concerning the utilization of multimodal data in federated learning.
• Developed a categorization framework for MFL methods (congruent vs. incongruent).
• Surveyed current applications and benchmarks for MFL.

Conclusions:

• MFL is a critical emerging area for enhancing federated learning systems.
• Further research is needed to address the fundamental challenges and unlock the potential of MFL.
• Standardized benchmarks and applications are crucial for MFL advancement.