A multimodal vision-language model for generalizable annotation-free pathology localization.

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Computer Vision

Background:

• Current deep learning models for medical image analysis require extensive expert annotations.
• These models often exhibit limited generalization in real-world clinical settings.
• Annotation requirements pose a significant bottleneck in developing robust AI for pathology detection.

Purpose of the Study:

• To introduce a generalizable vision-language model for annotation-free pathology localization (AFLoc).
• To overcome the limitations of expert annotation dependency in existing deep learning models.
• To adapt AI models to diverse pathological presentations without manual image labeling.

Main Methods:

• AFLoc utilizes multilevel semantic structure-based contrastive learning.
• This method aligns medical concepts with image features across multiple granularities.
• The model was trained on chest X-ray image-report pairs and validated on diverse external datasets.

Main Results:

• AFLoc achieved superior performance in annotation-free localization and classification compared to state-of-the-art methods.
• The model demonstrated robust generalization across various medical imaging modalities, including histopathology and retinal images.
• AFLoc surpassed human benchmarks in localizing specific pathological conditions.

Conclusions:

• AFLoc significantly reduces the need for expert annotations in pathology detection.
• The model shows high generalizability and applicability in complex clinical environments.
• This approach holds promise for advancing AI-driven diagnostic tools in healthcare.