Related Experiment Video
Updated: Aug 5, 2026
07:13
Multimodal Cross-Device and Marker-Free Co-Registration of Preclinical Imaging Modalities
Published on: October 27, 2023
MerMED-FM: Multimodal, Multi-Disease Medical Imaging Foundation Model
Yang Zhou1, Chrystie Wan Ning Quek2, Jun Zhou3
1School of Computer Science, Wuhan University, Wuhan, China; Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A∗STAR), Singapore.
The Lancet. Digital Health
|July 27, 2026
Summary
A new AI model, MerMED-FM, can interpret diverse medical imaging across specialties with high accuracy, even with limited labeled data. This versatile foundation model shows strong performance in radiology, histopathology, and ophthalmology.
Area of Science:
- Artificial Intelligence in Medical Imaging
- Multimodal Medical Data Analysis
- Foundation Models in Healthcare
Background:
- Current AI models for medical imaging are often limited to single modalities and diseases, leading to inconsistent clinical accuracy.
- Training these AI models requires extensive, well-labeled datasets, which are expensive and time-consuming to create.
- There is a need for AI models that can interpret diverse imaging modalities across specialties while maintaining high performance.
Purpose of the Study:
- To develop and evaluate a multimodal, multi-disease artificial intelligence (AI) foundation model for medical imaging interpretation.
- To assess the model's ability to maintain robust performance across various imaging modalities and specialties.
- To investigate the effectiveness of self-supervised learning and a memory module in training such a versatile AI model.
Main Methods:
- Developed the Multimodal, Multi-Disease Medical Imaging Foundation Model (MerMED-FM) using self-supervised learning and a memory module.
- Pretrained MerMED-FM on approximately 3.3 million unlabelled medical images from 12 specialties and seven imaging modalities.
- Fine-tuned, validated, and evaluated the model on 26 public and 5 private datasets across radiology, histopathology, and ophthalmology, comparing it against other foundation models.
Main Results:
- MerMED-FM demonstrated strong performance across all tested modalities with only 10% labeled data.
- Achieved high area under the receiver operating characteristic curve (AUROC) values, including 0.962 for optical coherence tomography (OCT) and 0.908 for histopathology.
- The model showed robust interpretation capabilities across chest x-rays, CT, ultrasound, histopathology, colour fundus photography (CFP), OCT, and dermatoscopy.
Conclusions:
- MerMED-FM shows potential as a highly adaptable and versatile cross-specialty foundation model for medical imaging.
- The model enables robust interpretation of diverse medical imaging data, overcoming limitations of single-modality and single-disease AI.
- This approach offers a promising direction for developing efficient and effective AI solutions in medical diagnostics.
