Knowledge-enhanced visual-language pre-training on chest radiology images.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Multi-modal foundation models show promise in vision and language tasks but face limitations in specialized medical domains.
• Medical AI requires deep domain knowledge and struggles with fine-grained diagnostic tasks.
• Existing models often lack the necessary medical expertise for accurate interpretation of clinical data.

Purpose of the Study:

• To introduce Knowledge-enhanced Auto Diagnosis (KAD), a novel approach for medical vision-language pre-training.
• To leverage existing medical domain knowledge to enhance the performance of AI models in analyzing chest X-rays and radiology reports.
• To evaluate KAD's effectiveness in both zero-shot and few-shot learning scenarios for medical diagnosis.

Main Methods:

• Developed KAD, integrating medical domain knowledge into vision-language pre-training.
• Utilized paired chest X-ray images and radiology reports for model training.
• Evaluated KAD on four independent external X-ray datasets.

Main Results:

• KAD achieved zero-shot performance comparable to fully supervised models.
• KAD demonstrated statistically significant superiority over expert radiologists in diagnosing three out of five pathologies.
• In few-shot settings, KAD outperformed all previously existing approaches.

Conclusions:

• KAD effectively integrates domain knowledge into medical AI, enhancing diagnostic capabilities.
• The model shows strong potential for clinical application, offering expert-level performance with minimal or no labeled data.
• KAD represents a significant advancement in applying foundation models to complex medical tasks.