Unity in Diversity: Collaborative Pre-training Across Multimodal Medical Sources

Area of Science:

• Biomedical Informatics
• Artificial Intelligence in Medicine
• Data Science

Background:

• Pre-trained models are crucial for biomedical tasks but limited by narrow data sources.
• Data scarcity and restricted applicability hinder current model efficacy.
• Bridging diverse medical data sources is essential for advancing AI in healthcare.

Purpose of the Study:

• To introduce Medical Cross-Source Pre-training (MEDCSP), a novel strategy to unify and leverage multimodal medical data from disparate sources.
• To address the limitations of data scarcity and limited downstream task applicability in current pre-trained biomedical models.
• To establish a foundation for cross-source modeling in the medical domain.

Main Methods:

• Developed MEDCSP, a pre-training strategy employing modality-level aggregation to unify patient data within sources.
• Utilized temporal information and diagnosis history to capture cross-source patient correlations (explicit and implicit).
• Conducted experiments using 6 modalities from 2 real-world medical datasets.

Main Results:

• MEDCSP demonstrated effectiveness in a cross-source modeling approach.
• Evaluated MEDCSP on 4 downstream tasks against 19 baseline models.
• The strategy successfully integrated and utilized data from multiple sources and modalities.

Conclusions:

• MEDCSP represents a significant advancement in pre-training for biomedical AI, addressing data limitations.
• The proposed method effectively bridges multimodal medical data gaps, enhancing model generalizability.
• This work is a foundational step towards robust cross-source medical data modeling.