Multimodal data fusion in neuroscience: promises, challenges and future directions

Area of Science:

• Neuroscience
• Biomedical Data Science

Background:

• Single modality analysis has limitations in capturing complex biological systems.
• Multimodal fusion integrates diverse data sources (neuroimaging, biomics, clinical phenotypes, text) to leverage shared and complementary information.

Purpose of the Study:

• To systematically review methods for fusing heterogeneous multimodal biomedical data, focusing on neuroscience applications.
• To discuss the strengths, limitations, and applications of these fusion strategies.
• To identify challenges and future directions in multimodal biomedical data fusion.

Main Methods:

• Systematic literature review of 302 research articles on multimodal fusion methods.
• Analysis of fusion strategies applied to neuroimaging, biomics, clinical phenotypes, and text data.
• Examination of applications across scientific research and clinical practice.

Main Results:

• Multimodal fusion offers significant advantages over single modality approaches.
• Various fusion methods exist with distinct strengths and limitations.
• Applications span from fundamental neuroscience research to clinical decision-making.

Conclusions:

• Multimodal fusion holds transformative potential for neuroscience.
• Future research should focus on model generalization, interpretability, data limitations, and unified platforms.
• Developing multimodal foundational models is crucial for bridging research and application.