The Application of Foundation Models in Radiology: Bridging Images, Reports, and Beyond

Area of Science:

• Radiology
• Artificial Intelligence
• Medical Imaging

Background:

• Artificial intelligence (AI) in radiology is evolving towards foundation models.
• These models learn universal representations from large-scale imaging and multimodal clinical data.

Purpose of the Study:

• To review the technical components of foundation models in radiology.
• To summarize recent advancements in image-only, chest X-ray image-report, and other image-report models.
• To discuss evaluation strategies, clinical validation, and future directions.

Main Methods:

• Review of technical components: self-supervised encoders, fusion modules, task-specific decoders.
• Categorization of recent work: image-only, chest X-ray image-report, and other image-report models.
• Discussion of evaluation strategies: vision-centric, language-centric, benchmark-based metrics, and clinical validation.

Main Results:

• Foundation models enable robust transfer learning with minimal annotation through self-supervised learning on unlabeled radiology scans.
• Joint visual and textual embeddings are achieved using large chest X-ray-reported corpora.
• Multimodal fusion integrates volumetric images with structured reports or clinical metadata for diverse applications.

Conclusions:

• Foundation models represent a significant advancement in AI for radiology.
• Future directions include multimodal integration and human-AI collaboration for personalized radiology.
• Addressing persistent challenges is crucial for clinical application and validation.