Lessons learned from RadiologyNET foundation models for transfer learning in medical radiology

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Deep Learning

Background:

• Medical deep learning models require extensive annotated data, which is scarce and costly to acquire.
• Foundation models pretrained on large datasets offer a potential solution to data scarcity and generalization issues.

Purpose of the Study:

• To evaluate the efficacy of a custom medical dataset, RadiologyNET, for pretraining foundation models.
• To compare RadiologyNET-pretrained models against ImageNet-pretrained and randomly initialized models across diverse medical tasks.
• To provide guidelines for utilizing foundation models in medical AI and release pretrained models.

Main Methods:

• Pretrained several popular deep learning architectures using the 1.9 million image RadiologyNET dataset.
• Evaluated model performance on segmentation, regression, binary classification, and multiclass classification tasks.
• Compared RadiologyNET and ImageNet foundation models against random initialization.

Main Results:

• RadiologyNET-pretrained models performed comparably to ImageNet models, with benefits in resource-limited scenarios.
• ImageNet models demonstrated strong performance when fine-tuned with sufficient data.
• Modality diversity in pretraining impacted performance variably across tasks.

Conclusions:

• Foundation models pretrained on RadiologyNET offer a valuable alternative to ImageNet models for medical AI.
• Aligning pretraining data modality with downstream tasks is crucial for optimal performance.
• The study provides practical guidelines and publicly released models to advance medical AI research.