BioFuse: an embedding fusion framework for biomedical foundation models.

Area of Science:

• Biomedical informatics
• Artificial intelligence in healthcare
• Machine learning for medical imaging

Background:

• The biomedical field increasingly uses pre-trained foundation models for specialized tasks like radiology and histopathology.
• Integrating diverse foundation models presents challenges in compatibility and feature fusion for comprehensive data analysis.
• Existing methods struggle to effectively combine embeddings from multiple models for enhanced performance.

Purpose of the Study:

• To introduce BioFuse, an open-source framework for generating optimized biomedical embeddings.
• To address challenges in model compatibility and feature fusion when integrating multiple foundation models.
• To create task-specific embeddings by leveraging a pool of state-of-the-art foundation models.

Main Methods:

• Utilized a pool of 9 state-of-the-art foundation models.
• Employed grid search to identify optimal model combinations for embedding fusion.
• Fused embeddings through vector concatenation for downstream classification tasks.
• Evaluated performance on the MedMNIST+ benchmark using XGBoost classifier.

Main Results:

• BioFuse achieved state-of-the-art (SOTA) Area Under the Curve (AUC) in 5 out of 12 MedMNIST+ datasets.
• Maintained near-SOTA performance across most remaining datasets.
• Discovered unexpected cross-modal capabilities, with models excelling in non-native imaging modalities.
• Demonstrated superior performance compared to several existing embedding fusion methods.

Conclusions:

• BioFuse offers an effective framework for optimizing biomedical embeddings by fusing multiple foundation models.
• The framework demonstrates strong performance and reveals potential for cross-modal learning in biomedical data.
• BioFuse's high-level API and extensible architecture facilitate immediate deployment and future model integration.