Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Vision

Background:

• Deep neural networks (DNNs) are vital for medical image analysis, but training data scarcity poses a significant challenge.
• Pre-trained contrastive image-text models (e.g., CLIP) are adapted for medical tasks, enabling zero-shot pathology classification by leveraging chest X-ray images and reports.
• Existing methods often overlook the multi-labeled nature of medical data, using standard contrastive learning objectives.

Purpose of the Study:

• To propose a new fine-tuning strategy to improve zero-shot pathology classification performance in medical imaging.
• To address the limitations of current contrastive learning approaches by accounting for multi-labeled medical image-report pairs.
• To enhance the generalizability of models to out-of-domain datasets without requiring additional training or external knowledge.

Main Methods:

• Developed a novel fine-tuning strategy incorporating positive-pair loss relaxation and random sentence sampling.
• Applied the proposed method to fine-tune pre-trained contrastive image-text encoders using chest X-ray datasets and reports.
• Evaluated the approach across four diverse chest X-ray datasets and three different pre-trained models.

Main Results:

• Achieved consistent improvements in overall zero-shot pathology classification across all tested datasets and models.
• Demonstrated an average macro Area Under the Receiver Operating Characteristic Curve (AUROC) increase of 4.3%.
• Outperformed existing state-of-the-art methods and marginally surpassed board-certified radiologists in zero-shot classification for specific CheXpert pathologies.

Conclusions:

• The proposed fine-tuning strategy effectively enhances zero-shot pathology classification in medical imaging, particularly for multi-labeled data.
• The method offers a robust and generalizable solution applicable to various pre-trained models and datasets without external data reliance.
• This approach represents a significant advancement, showing potential to aid in radiological diagnosis and improve efficiency.