MIRAGE: Medical image-text pre-training for robustness against noisy environments

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Computer Vision

Background:

• Contrastive vision-language pre-training (VLP) models excel on large general datasets.
• Medical datasets are often small and noisy due to high data collection and annotation costs.
• Existing methods struggle with noisy medical data, leading to overfitting and poor representations.

Purpose of the Study:

• To introduce MIRAGE, a framework for robust medical image-text pre-training.
• To address challenges posed by mismatched false positives and semantically related false negatives in noisy medical datasets.
• To improve the performance of VLP models in the medical domain.

Main Methods:

• Developed MIRAGE, a novel framework for medical image-text pre-training.
• Introduced an optimal transport-based contrastive loss to identify and mitigate noisy samples.
• Implemented an adaptive gradient balancing strategy to reduce the impact of noisy gradients.

Main Results:

• MIRAGE demonstrated superior performance across six tasks and 14 datasets.
• The framework significantly outperformed existing state-of-the-art methods.
• Analyses confirmed the effectiveness of each component in handling noisy synthetic data.

Conclusions:

• MIRAGE offers a robust solution for medical image-text pre-training with noisy data.
• The proposed optimal transport loss and gradient balancing are key to overcoming data limitations.
• This framework advances the application of VLP in medical AI.